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Abstract 

Total  Quality  (TQ)  is  a  philosophy  that  is  currently  being  incorporated  by  organizations 
throughout  America.  TQ  philosophers  maintain  that  Total  Quality  can  be  achieved  only  if 
organizations  change  their  cultures,  including  the  traditional  view  toward  the  rote  of  the 
performance  appraisal.  Theses  experts  maintain  that  the  traditional  performance  appraisal 
doesn't  promote  process  improvement,  empowerment  and  team  building  as  TQ 
prescribes. 

This  thesis  is  a  Delphi  study  of  the  performance  appraisal  practices  of  Malcolm  Baldrige 
National  Quality  Award  winning  companies.  The  study  attempts  to  describe  the 
performance  appraisal  practices  of  10  companies  who  have  successfully  implemented  TQ. 
The  existence  of  traditional  and  TQ  appraisal  charaaeristics  is  measured  and  the 
performance  appraisal  techniques  of  the  organizations  are  benchmarked.  The  study 
determii^  that,  with  little  exceptioii,  these  companies  still  employ  many  traditional 
approaches  to  their  performance  appraisals,  but  have  altered  some  practices  by 
incorporating  new  techniques  to  promote  process  improvement,  employee  empowerment 
and  team  building.  The  unique  practices  and  charaaeristics  of  these  organizations  are 
presented. 
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A  DELPHI  STUDY  OF  PERFORMANCE  APPRAISAL  PRACTICES  OF 


MALCOLM  BALDRIGE  AWARD  WINNING  ORGANIZATIONS 


I:  Introduction 


Background 

Total  Quality  Management  (TQM)  is  the  single  most  important  management 
methodology  available  today  to  achieve  and  maintain  a  competitive  edge  (Spenley, 

1992: 10).  Total  Quality  Management  seeks  to  achieve  total  customer  satisfaction  by 
operating  on  the  principle  of  long-term,  gradual  and  continuous  improvement  of  an 
organization’s  processes.  Traditional,  Western  management,  on  the  other  hand,  places 
emphasis  on  short-term  rewards,  resuits  rather  than  processes,  and  parochial  rather  than 
systemic  views  towards  operations  (Liberatore,  1993:62). 

It  is  critical  that  organizations  who  aspire  to  implement  TQM  realize  that  to  do  so 
requires  a  significant  cultural  change.  Culture,  or  corporate  systems,  can  be  defined  as 
“the  procedures  and  policies  that,  when  combined  with  employees,  equipment,  methods, 
materials,  suppliers  and  customers  become  the  dynamic  processes  of  doing  business” 
(Liberatore,  1993:61).  Successful  incorporation  of  TQM  requires  those  at  all  levels  of  an 
organization  to  realize  that  TQM  is  a  different  methodology  and  that  the  elements  that 
mold  the  culture,  or  the  corporate  system,  must  change  in  order  to  facilitate  the  attainment 
ofTQM, 
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General  Issue 


To  be  abb  to  make  the  transition  to  TQM,  Saylor  submits  that  all  people  must  be 
“empowered  and  encouraged  to  be  creative  in  an  effort  to  achieve  continuous 
improvement  in  their  workplaces.  They  must  be  allowed  to  make  any  changes  that  are 
necessary  to  perform  the  work  and  improve  the  system”  (Saylor,  1992:27).  This  cultural 
change  of  employee  empowerment  and  focus  on  long-term  improvement  must  begin  with 
the  leadership  of  the  organization. 

An  element  which  leaders  must  consider  in  establishing  the  TQM  environment  is  the 
way  employees  approach  their  jobs,  and  an  element  which  can  impact  the  way  members 
^proach  their  jobs  is  the  performance  appraisal.  ITie  performance  appraisal  is  a 

technique  for  measuring  «npioyee  actions  against  standards  of  performance  iPlachy. 

* 

1991 :57).  Because  the  performance  appraisal  identifies  standards  of  performance,  the 
organization  communicates  to  its  employees,  via  the  appraisal,  the  behaviors,  actions,  and 
performance  elements  that  are  looked  upon  favorably  by  the  organization.  For  this 
reason,  it  is  the  opinion  of  the  researchers  tliat  the  contents  and  purposes  of  the 
performance-appraisal  can  have  an  impact  on  an  organization' s  corporate  culture  and  its 
transformation,  into  the  TQM  environment. 

Problem  Area 

There  are  those  who  believe  that  the  use  of  performance  appraisals  is  not  consistent 
with  the  principles  behind  TQM.  W.  Edwards  Deming,  for  example,  cites  evaluation  of 
performance  by  merit  rating  or  annual  review  as  one  of  Western  management’s  Seven 


Deadly  Diseases  which  prevents  organizations  from  entering  the  TQM  environment. 
Deming  maintains  that  evaluating  performance  by  these  means  “nourishes  short-term 
performance,  annihilates  long-term  planning,  demolishes  teamwork,  and  nourishes  rivalry 
and  politics”  (Deming,  1 986: 101). 

According  to  Bowman,  performance  ratings  should  be  abolished  because  they  conflict 
with  the  search  for  quality.  The  problem  with  the  performance  appraisal  is  that  it  focuses 
on  the  end  results  and  assumes  employees  are  responsible  for  those  results.  TQM,  on  the 
other  hand,  re  ignizes  that  ail  work  is  part  of  an  overall  process  (Bowman,  1 994;  131). 
This  means  that  there  is  an  understanding  that  the  subumts  of  an  organization  interact  as  a 
system,  thus  the  perfonnance  of  a  subunit  (as  well  as  the  individuals  within  it)  cannot  be 
accurately  evaluated  as  an  independent  function.  The  focus  needs  to  move  from  the 
individual  to  the  overall  system  since  over  95  percent  of  all  quality  problems  are  system- 
related,  not  caused  by  the  individual  (Scholtes,  1993:351).  With  regard  to  the 
performance  appraisal,  Scholtes  supports  these  points  by  maintaining  that  the 
performance  appraisal  is  based  on  the  erroneous  assumption  that  the  reviewer,  or  rater, 
can  distinguish  an  individual’s  contribution  from  the  other  influences  contributed  by  the 
overall  system.  The  system  is  always  dynamic  and  variable  and  those  forces  within  the 
system  impact  the  performance  of  the  individual.  (Scholtes,  1993:356) 

Research  Approach 

This  study  applies  the  Delphi  technique  methodology  to  describe  how  organizations 
who  have  been  nationally  recognized  for  TQM  e.xcellence  make  use  of  the  perfonnance 
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^praisal.  All  ten  organizations  who  participated  in  this  research  are  former  winners  of 
the  Malcolm  Baidrige  National  Quality  Award.  In  1987,  the  Reagan  Administration 
established  the  Malcolm  Baidrige  National  Quality  Award  to  “promote  quality  awareness, 
understand  the  requirements  for  quality  excellence,  and  share  information  about  quality 
strategies”  (Nakhai  and  Neves,  1994:33).  Organizations  throughout  the  nation  have 
realized  the  importance  of  qjplying  the  Baidrige  criteria  to  their  business  processes.  In 
fact,  oi^anizations  requested  over  230,000  copies  of  the  Baidrige  guidelines  in  1992 
(Brown,  1993:2). 

The  Delphi  technique  is  a  method  of  eliciting  and  refining  group  judgments. 
Anonymous  responses  of  group  members,  usually  comprised  of  experts  in  a  certain  area, 
are  “obtained  by  a  systematic  exercise  conducted  during  the  course  of  several  iterations” 
(Dalkey,  1969:v).  The  Delphi  methodology  is  appropriate  for  this  study  for  two  main 
reasons.  First,  there  are  very  few  winners  of  the  Baidrige  Award  to  date.  Since  there  are 
few  participants  in  this  research,  it  is  possible  to  gather  the  neeed  data  &om  the 
participants  via  written  correspondence  over  several  iterations.  Second,  since  this  study 
investigates  one  element  of  an  organization,  the  performance  appraisal,  it  is  possible  to 
solicit  the  participation  of  one  expert  in  this  area  from  each  organization. 

It  is  the  researchers’  intent  that  the  information  derived  fixjm  this  study  will  be  used  by 
organizations,  including  those  in  the  Department  of  Defense,  to  provide  a  benchmark  for 
their  performance  appraisal  practices  through  comparison  with  the  practices  that  are  used 
by  organizations  demonstrating  quality  improvement.  Benchmarking  is  the  process  of 
continually  comparing  and  measuring  an  organization’s  processes  with  those  of  business 
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leaders  to  gain  information  which  will  help  the  organization  take  action  to  improve  its 
performance  (American  Productivity  and  Quality  Center,  1993:4).  Because  the  process  of 
benchmarking  involves  continuous  comparison  and  measurement  it  is  a  methodology 
which  is  well-suited  for  use  in  the  TQM  environment.  Delphi  and  benchmarking  will  be 
discussed  in  more  detail  in  Ch^ter  m. 

Research  Problem 

Total  Quality  experts  such  as  W.  Edwards  Deming  maintain  that  the  performance 
appraisal  is  incompatible  with  TQM.  This  viewpoint  seems  to  be  in  conflict  with  the 
majority  of  the  available  literature  on  this  topic  which  views  the  performance  appraisal  as 
a  necessary  tool  to  ensure  desired  performance  measures  are  being  met.  The  objective  of 
this  research  is  to  determine  the  extent  to  which  the  performance  appraisals  of  successful' 
TQ  organizations  are  in  agreement  with  Deming’s  assertion  that  the  appraisal  is  one  of 
Western  management’s  Seven  Deadly  Diseases. 

Investigative  Questions 

This  research  uses  the  Delpld  technique  to  answer  the  following  investigative 
questions: 

1 .  Based  on  the  available  literature,  what  are  the  elements  and  uses  of  performance 
appraisals? 

2.  What  are  the  differences  between  the  traditional  and  the  TQ-oriented  approaches  to 
the  performance  appraisal? 

3.  What  characteristics  of  the  performance  appraisal  are  currently  in  use  in  the  TQM 
environment? 

4.  Which  characteristics  are  viewed  as  mcst  significant  by  the  organizational  experts? 


Limitations  and  Scope  of  Research 

'i  he  scope  of  this  research  is  limited  to  the  performance  ^praisal  systems  of  Malcolm 
Baldrige  Award  winners.  It  does  not  include  other  elements  of  these  organizations’ 
human  resource  man^ement  programs,  nor  does  it  address  other  elements  of  their  overall 
business  practices.  Additionally,  this  research  is  net  intended  to  prescribe  how  to 
implement  a  performance  appraisal  system.  Rather,  it  is  intended  to  offer  a  description  of 
the  performance  appraisal  practices  of  successful  Total  Quality  (TQ)  organizations. 

Because  this  research  is  aimed  at  identifying  the  pjerformance  appraisal  practices  of 
Malcolm  Baldrige  Award  winners,  it  may  provide  a  benchmark  for  other  organizations 
wishing  to  evaluate  and  improve  their  own  appraisal  systems.  It  is  the  opinion  of  the 
researchers  that  the  information  derived  from  this  study  could  be  of  benefit  to  most 
organizations  who  have  a  desire  to  follow  the  TQM  movement 

Summaty 

This  chapter  provided  the  basic  rationale  behind  this  research.  Since  the  elements  and 
implementation  of  an  organization’s  performance  appraisal  can  have  an  impact  on 
shaping  employee  behavior  and  corporate  culture,  most  organizations  desiring  to  enter  the 
TQM  environment  including  the  United  States  Air  Force  and  other  branches  of  the 
Department  of  Defense,  can  benefit  by  learning  the  practices  of  the  best  TQ  companies  in 
America.  Chapter  II  reviews  the  literature  pertaining  to  issues  such  as  the  use,  design, 
and  elements  of  tf.e  performance  appraisal,  zind  Chapter  III  discusses  the  Delphi  technique 
and  benchmarking  methodologies  relevant  to  this  study.  Chapter  IV  analyzes  the  answers 
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fix)in  the  Delphi  study  as  well  as  the  answers  to  the  questions  posed  to  the  experts 
themselves.  Chapter  V  discusses  the  conclusions,  recommendations  for  further  study, 
and  any  further  limitations. 
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II:  Literature  Review 


Introduction 

This  literature  review  is  partitioned  into  two  major  segments.  The  first  section 
addresses  general  literature  pertaining  to  performance  appraisal  systems.  This  includes 
literature  on  the  purposes  of  performance  appraisal  systems,  elements  of  the  appraisal,  the 
goals  of  performance  appraisal  systems,  and  rater  accuracy  and  training.  The  second 
section  revic’vs  more  recent  literature  regarding  the  performance  appraisal  and  its 
compatibility  with  the  Total  Quality  Management  philosophy. 

Performance  Appraisal  Systems 

A  definition  of  the  performance  appraisal  which  the  researchers  have  chosen  for  the 
purposes  of  this  research  is  “is  a  teclmique  for  measuring  employee  actions  gainst 
standards  of  acceptable  performance"  (Plachy,  1983:57).  This  section  addresses  purposes 
of  ^praisal  systems,  elements  of  the  appraisal,  goals  of  appraisal  systems  and  rater 
accuracy  and  training. 

Appraisal  Purposes  and  Elements.  The  performance  appraisal  is  a  widely  used  tool 
in  American  businesses  and  other  organizations.  It  is  used  for  a  variety  of  purposes  such 
as  feedback,  personnel  actions,  promotion,  wages  and  salary,  layoffs,  employee 
development,  and  placement  The  critical  requirement  of  an  effective  performance 
appraisal  is  that  its  elements  be  linked  to  both  the  goals  of  the  organization  and  employee 
development,  (Sauser,  1980:12) 
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In  1984  the  American  Management  Association  surveyed  588  members  of  its  Human 
Resources,  Finance,  Marketing  and  Information  Systems  divisions  to  identify  the  most 
common  purposes  for  the  performance  appraisal  (Moen,  1989:62).  The  results  of  this 
survey  are  shown  in  Table  1  The  results  can  be  broken  into  three  sections.  According  to 
this  survey,  the  primary  purpose  the  performance  appraisal  serves  is  determining  monetary 
compensation.  Although  secondary  to  compensation-related  purposes,  both  counseling 
and  training  and  development  achieved  about  the  same  response.  Interestingly,  less  than 
half  of  the  respondents  said  they  used  the  appraisal  for  promotion  purposes. 


Table  1,  Common  Purposes  for  the  Performance  Appraisal  (Moen,  1989:62) 


1  Purpose 

Percent  of  those  responding 

85.6% 

Counseling 

65.1 

Training  and  Development 

64.3 

Promotion 

45.3 

Staff  Planning 

43  1 

Retention/Discharge 

30.3 

Validation  of  Selection  Technique 

17.2 

Five  basic  elements  of  an  effective  appraisal  include  1)  having  measurable  objectives 
for  each  position;  2)  making  performance  appraisal  a  continuous,  year-round  objective; 

3)  linking  the  proems  to  positive  development,  4)  educating  appraisers  about  performance 
appraisal  purposes  and  practices  (Sahl,  1990:55);  and  5)  giving  the  employee  the 
opportunities  to  review  and,  if  desired,  appeal  the  performance  rating  (Daley,  1992  :40) 
Although  some  research  identifies  other  elements,  most  include  these  five  Additior  ally, 
recent  court  decisions  have  set  legal  bounds  on  various  aspects  of  performance  appraisals. 
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They  include:  1)  the  rating  method  must  be  shown  to  be  job-related,  2)  the  content  of  the 
rating  method  must  be  developed  through  job  analysis,  3)  raters  must  be  able  to 
consistently  observe  the  ratee’s  performance;  4)  ratings  must  not  be  based  on  subjective 
or  vague  factors;  5)  racial,  sexual,  or  other  biases  may  not  influence  ratings,  and  6) 
ratings  must  be  collected  and  scored  under  standardized  conditions  (Holley, 

1976:  463), 

With  regard  to  the  types  of  data  used  in  the  performance  appraisal,  Sauser  describes 
two  general  categories  of  data  that  raters  commonly  use  to  evaluate  employees— hard  and 
soft  criteria.  Hard  criteria  are  contained  in  the  records  of  the  organization.  These  include 
quantity  and  quality  of  employee  output,  absences,  overtime,  reprimands,  and  so  forth. 
Hard  criteria,  which  on  the  surface  may  seem  reliable,  may  ttot  be  because  of  factors  such 
as  inaccurate  or  biased  record  keeping,  or  because  the  employee’s  output  and  other 
performance  metrics  that  are  used  as  a  basis  for  evaluation  may  be  influenced  by  factors 
that  are  out  of  his  or  her  control.  These  factors  can  include  seasonal  variation, 
inconsistencies  in  input  materials,  and  problems  within  the  overall  processes  of  the 
organization.  Additionally,  the  more  the  employee  performs  in  a  supervisory  capacity,  the 
less  hard  criteria  are  directly  attributable  to  him  or  her.  Soft  criteria,  on  the  other  hand, 
arc  usually  more  subjective  in  nature  and  arc  typically  found  in  a  scale  or  mark-the-box 
format.  These  include  factors  such  as  attitude,  communication  skills,  and  leadership 
ability.  These  criteria  are  dependent  upon  the  rater’s  impressions  and,  as  such,  are  also 
subject  to  bias  and  distonion  (Sauser,  1980  13) 
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Goals  of  Perfoi  maDce  Appraisal  Systems.  The  explicit  goal  or  purpose  of  a 
performance  appraisal  system  is  to  accurately  measure  employee  performance  (Plachy, 
1983:58).  The  implicit  purposes  of  a  performance  appraisal  system  are  less  evident. 
Organizational  effectiveness  is  the  ultimate  purpose  for  an  organization  To  accomplish 
this  end,  performance  appraisal  systems  have  two  general  categories  of  purposes.  These 
include  administrative  and  behavioral  purposes. 

Administrative  purposes  are  those  actions  the  organization  takes  to  improve  employee 
performance  (Pierson,  1980: 145).  Administrative  purposes  include  granting  or 
withholding  raises  and  identuymg  those  who  deserve  promotion,  or  those  who  need 
training.  These  administrative  applications  affect  behavior  based  on  their  power  to  reward 
or  punish  an  individual.  Most  performance  appraisal  systems  are  used  in  this 
manner. 

Behavioral  purposes,  on  the  other  hand  attempt  to  encourage  the  individual  to  bring 
change  about  himself  without  the  threat  of  organizational  action.  Dr.  Ralph  M.  Pierson 
says,  “Even  though  behavioral  purposes  are  those  the  individual  must  effect,  the 
organization  has  a  great  deal  of  interest  in  them.  Its  concern  is  that  employee  job  behavior 
changes  for  the  better"  (1980: 146)  Both  behavioral  and  administrative  purposes  are 
related  to  one  another  but  they  are  distinct.  They  both  have  an  overall  goal  to  increase 
organizational  effectiveness,  but  the  means  to  accomplish  these  ends  are  different 
Appraisal  systems  must  consider  each  of  these  purposes  separately  to  design  an  effective 
performance  appraisal  system  (Pierson,  1980: 145) 
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As  mentioned  previously,  the  use  of  performance  appraisal  systems  by  organizations  is 
quite  common.  Studies  have  shown  that  managers  are  generally  satisfied  with  their 
performance  appraisal  systems  and  they  feel  these  systems  help  the  company  meet 
organizational  goals  (Guzzo  and  others.  1985  277).  In  fact,  managers  attribute  several 
factors  to  effective  performance  appraisal  systems.  These  include  improving  performance 
and  productivity,  identifying  training  deficiencies,  and  identifying  promotable  employees 
(Dmitroff  and  Dwyer,  1976:353).  Whatever  its  intended  purpose,  the  well-designed 
performance  appraisal  system  can  contribute  to  an  organization's  effectiveness  (Plachy, 
1991:51). 

General  Types  of  Performance  Appraisal  Systems.  Although  there  are  many 
different  types  of  performance  appraisals  used  by  organizations,  most  can  be  grouped  into 
four  general  categories  based  either  on  their  method  of  appraising  or  their  method  of 
ranking  employees. 

Appraisal  Methods.  The  first  two  categories  of  performance  appraisal  systems  are 
the  traditional  and  the  collaborative.  The  traditional  approach  uses  a  quantitative  scale  to 
rate  employees’  performance  compared  to  organizational  objectives  (Taylor  and  Zawacki, 
1976:290).  The  traditional  approach  also  uses  one-way  communication,  where  the 
supervisor  or  the  organization  sets  the  standards.  In  other  words,  standards  of 
performance  are  not  mutually  agreed  upon  between  supervisor  and  employee  Proponents 
of  this  approach  argue  that  traditional  rating  systems  used  within  this  approach  are 
effective  and  free  from  difficulties  when  used  with  reasonable  judgment  (Taylor  and 
Zawacki,  1976.292). 
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Unlike  the  traditionai  approach  which  is  concerned  with  performance  in  comparison  to 
a  standard,  the  collaborative  approach  advocates  a  system  where  the  human  development 
of  the  employee  is  the  major  concern.  Its  proponents  argue  for  more  communication  with 
subordinates.  This  communication  focuses  on  joint  goal  setting.  Employees  are  much 
more  involved  under  the  collaborative  approach  than  under  the  traditional  approach, 
which  ignores  the  effect  of  the  performance  appraisal  on  the  individual  (Taylor  and 
Zawacki,  1976:291). 

Ranking  Methods,  The  other  two  categories  of  performance  appraisal  systems  are 
based  on  their  method  of  ranking  employees.  These  categories  are  the  norm-referenced 
ranking  system  and  the  criterion-referenced  approach.  The  norm-referenced  system  is 
"where  a  person  is  compared,  not  with  respect  to  performance  objectives,  but  rather  with 
reference  to  how  the  average  or  norm  performs  in  the  organization”  (Morano,  1979:306) 
This  ^proach  is  similar  to  curved  grading  scale  in  a  college  class.  Under  tl.e  curved 
grading  scale  approach,  students  are  rated  average  if  they  perform  at  the  level  of  the 
majority  of  the  students  in  the  class.  They  are  not  evaluated  based  on  preestablished 
criteria.  Rather,  evaluation  depends  on  how  eveiy'one  else  performs  Since  the  average 
may  differ  from  class  to  class,  one  shortfall  of  this  system  is  that  average  performance  in 
one  class  may  be  either  exceptional  or  failing  in  another  Another  shonfall  of  this 
approach  is  that  this  type  of  evaluation  may  not  comply  with  the  legal  requi''iment  of 
standardized  scoring  within  an  organization. 

Under  the  criterion-referenced  approach,  however,  people  are  compared  with  the 
performance  objectives  of  their  jobs  To  continue  the  classroom  analogy,  it  is 
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conceivafale  that  under  a  criterion-referenced  system,  all  students  in  a  classroom  could 
receive  an  "A"  and  equally  conceivable  that  all  could  receive  an  “F’  This  is  because 
performance  is  based  on  a  preestablished,  fixed  set  of  standards. 

When  applied  to  performance  appraisals,  the  norm-referenced  system  results  in  a 
majority  rating  of  average,  while  '\e  criterion-referenced  approach  could  result  in  a  high 
number  of  superior  or  inferior  ratings  of  employees  (Morano,  1979:307) 

Rater  Errors^  Accuracy,  and  Rater  Training.  One  of  the  main  criticisms  of 
performance  evaluations  is  that  they  are  subject  to  rater  errors  such  as  halo  and  leniency 
effects.  Halo  effect  occurs  when  a  rater  tends  to  allow  overall  assessments  of  a  ratee  to  be 
unduly  influenced  by  his  or  her  evaluation  of  jitst  a  few  factors.  For  example,  say  an 
employee  presents  a  good  personal  image  and  is  articulate.  A  rater  may  be  so  impressed 
by  these  attributes  that  his  or  her  evaluation  of  this  employee  may  be  based  solely  on  these 
criteria.  .Another  common  form  of  the  halo  effect  is  the  first  impression.  A  posirive  or  a 
negative  first  impression  can  influence  a  rater’s  opinion  of  an  employee  to  the  point  where 
the  employee’s  actual  performance  is  secondary  in  the  eyes  of  the  rater.  Additionally, 
outstanding  or  poor  performance  on  one  occasion  can  also  sway  a  rater’s  opinion  of  a 
subordinate.  Leniency  effect,  on  the  other  hand,  occurs  when  raters  tend  to  give  a 
disproportionaic  number  of  favorable  ratings  to  employees  (Holley,  1976  :458)  Leniency 
errors  tend  to  occur  more  frequently  under  the  criterion-referenced  approach. 

Cronbach  developed  five  performance  appraisal  accuracy  indicators  The  first, 
evaluation  accuracy  con.^:  i  -  "jf  the  average  of  a  rater’s  ratings  across  all  behaviors  and 
across  all  ratees,  thus  reflecting  how  he  or  she  interpreted  the  rating  scale.  Second, 
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differential  evaluation  describes  the  rater's  ability  to  judge  deviations  in  a  ratee's 
performance  from  his  or  her  normal,  overall  performance.  Third,  stereotype  accuracy 
describes  how  well  the  rater's  judgment  coincided  with  the  mean  overall  rating  for  all 
raters.  Fourth,  differential  accuracy  indicates  how  well  a  rater  discriminated  among 
dimensions,  or  elements  of  performance,  across  and  among  ratees.  Finally,  overall 
accuracy  is  the  overall  comparison  of  the  rater's  judgment  to  the  standard.  Again,  this 
implies  that  the  standard  is  known  or  can  be  determined  by  expert  raters  (Cronbach, 
1955:177-193). 

Rater  training  has  received  much  attention  by  researchers.  This  area  of  research  has 
been  concerned  with  the  impact  of  training  on  reducing  rater  errors  and  improving 
accuracy.  Smith  reviewed  24  studies  on  the  effects  of  rater  training  on  the  psychometric 
quality  and  accuracy  of  performance  reviews  (Smith,  1986:22-40).  By  synthesizing  the 
data.  Smith  found  that  certain  training  formats  were  effective  in  reducing  halo  and  leniency 
errors  while  others  were  not.  For  example,  lectures  were  found  to  be  as  effective  as  group 
discussion  in  reducing  halo  effects  but  were  ineffective  in  reducing  leniency  errors.  His 
reason  for  this  was  that  mere  rater  awareness  of  what  halo  errors  were  was  sufficient 
whereas  group  discussions  were  necessary  to  reduce  leniency  errors. 

It  is  important  to  have  a  performance  appraisal  system  that  strikes  a  balance  between 

promoting  the  overall  goals  of  the  organization  and  facilitating  employee  development. 

Instead  of  acting  as  an  incentive  to  stimulate  workers  to  greater  performance, 
many  performance  appraisals  actually  have  the  opposite  effect.  They  tend  to 
discourage  and  disgruntle  employees  when  the  latter  see  little  link  between 
performance  and  reward.  (Morano,  1979:307) 


Propcinents  of  the  Toud  Quality  Management  philosophy  tend  to  agree  with  this 
statement.  They  maintain  the  appraisal  is  being  used  as  a  tool  to  ensure  production  goals 
are  met  rather  than  a  tool  to  encourage  employee  improvement  and  development.  The 
following  sections  review  Total  Quality  Management  (TQM)  and  its  compatibility  with  the 
performance  appraisal. 

Total  Quality  Management  and  the  Performance  Appraisal 

TQM  Overview.  Over  the  past  decade,  the  Total  Quality  Management  movement  has 
been  embraced  by  organizations  throughout  the  United  States.  In  faa,  TQM  has  been  said 
to  be  the  single  most  important  management  methodology  available  today  to  achieve  and 
maintain  a  competitive  edge  (Spenley,  1992:10).  To  understand  the  view  many  TQM 
proponents  have  regarding  the  performance  appraisal’s  compaf’'  Jt\  with  TQM,  this 
section  outlines  the  principles  behind  the  TQM  philosophy  and  reviews  alternative 
methods  for  evaluating  employee  and  organizational  effectiveness. 

The  TQM  process  attempts  to  eliminate  traditional  W  astern  management  views  and 
practices  that  focus  on  short-term  results,  emphasize  the  product,  not  the  process,  and 
performance  evaluation  based  on  individual  performance.  TQM  is  based  on  the  principles 
of  long-term,  continuous  improvement,  emphasis  on  the  process  rather  than  on  the 
product,  and  the  importance  of  the  group  or  team  over  the  individual  (Liberatore, 

1993  62)  This  radical  depaiture  from  the  traditional  American  management  style  has 
presented  a  challenge  to  managers.  The  successfiil  implementation  of  TQM  requires 
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managers  at  all  levels  to  reexamine  their  roles  and  to  develop  new  skills  and  methods  in 
order  to  adopt  this  new  paradigm  (Canin,  1993:4). 

However,  it  has  been  easier  for  managers  to  encourage  TQM  awareness  than  it  has  to 
aiCcetsfuUy  create  a  cultural  shift  by  incorporating  TQM  throughout  the  organization. 

Tois  means  that  managers  have  learned  the  “rhetoric  of  Total  Quality  and  have  adopted 
programs  to  infuse  TQM  into  their  organizations,  but  few  have  appreciated  the  profoundly 
different  approach  it  requires  of  those  who  must  lead”  (Schoites,  1993:349).  This 
approach  is  different  because  TQM  places  importance  on  long-term,  continuous 
improvement  while  focusing  on  total  customer  satisfaction.  To  take  this  approach 
requires  management  to  assume  a  systems  perspective  of  the  organization.  In  other 
words,  all  departments  of  the  organization  must  be  viewed  as  being  interrelated  and 
interdependent  parts  of  an  overall  process  rather  than  being  separate  fimetions  (Cartin, 
1993:14). 

W.  Edwards  Denting,  arguably  the  pioneer  of  the  Total  Quality  movement,  developed 
his  14  Points  for  Continuous  Improvement,  which  can  be  considered  the  backbone  of 
TQM.  These  are  briefly  listed  in  Table  2  on  the  following  page.  A  more  detailed 
description  of  Denting’ s  14  Points  is  contained  iu  Appendix  A. 

In  addition  to  his  14  points.  Denting  identified  seven  deadly  diseases  that  plague 
American  orgaitizations  and  prevent  TQM  fi-om  being  fully  realized  (Walton,  1986:36, 
Denting,  1986  100).  They  are  briefly  listed  in  Table  3  on  the  following  page.  A  more 
detailed  description  of  these  seven  deadly  diseases  can  be  found  in  Appendix  B 


Table  2,  Denning’s  14  Points  (Deming,  1986:23-96) 


1 .  Create  constancy  of  purpose  and  manaeement  commitment 


2.  Adopt  the  new  philosoph 


3 .  Understand  Inspection 


4.  End  the  practice  of  awarding  business  based  on  price  tag 
alone 


5.  Improve  constantl 


6.  Institute  trainin 


7.  Institute  leader shi 


8.  Drive  out  fear 


9.  Break  down  barriers  between  staff  areas 


10.  Eliminate  slogans  and  targets  for  the  workplace 


11.  Eliminate  quotas  and  Management  by  Objectives  (MBO 


12.  Remove  barriers  to  pride  in  workmanshi 


13.  Institute  education  and  trainin 


14.  Take  action  to  accomplish  the  transformation 


Table  3,  Deming’s  Seven  Deadly  Dueases  (Deming,  1986:97-153) 


1 .  Lack  of  constancy  of  purpose 


2.  Emphasis  on  short-term  profits 


3.  Evaluation  by  performance,  merit  rating,  or  annual  review 


4.  Mobility  of  mar^gement 


5.  Running  a  company  on  visible  figures  alone 


6.  Excessive  medical  costs 


7.  Excessive  costs  of  warranty  fueled  by  lawyers  that  work 
on  contingency  fee  (Deming,  1986:23-96) 


Deadly  Disease  Number  Three,  For  the  purposes  of  this  research,  the  emphasis  is  on 
the  third  Deadly  Disease,  evaluation  by  performance,  merit  rating  or  annual  review. 
Deming  idencifies  several  problems  with  performance  appraisals.  First,  as  stated 
previously,  appraisals  foster  short-term  performance  rather  than  long-term  planning.  This 
is  because  appraisals  focus  on  short-term  results-oriented  performance  metrics  rather  than 
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contributions  to  improvement  on  a  systemic  level.  Second,  they  are  destructive  to  the 
person  being  reviewed,  since  the  traditional  carrot-and-stick"  evaluation  criteria  inhibit 
employees  from  presenting  a  different  point  of  view  or  making  process-improving 
suggestions  for  fear  that  by  doing  so,  it  may  appear  that  he  or  she  is  doubting 
management’s  abilities.  Third,  appraisals  are  detrimental  to  fostering  teamwork  as 
workers  compete  for  ratings  and  compensation.  Fourth,  employees  are  reluctant  to  help  a 
coworker  with  his  or  her  goals  be'^ause  it  takes  away  from  one’s  own  goal  achievement. 
Fifth,  appraisals  focus  on  the  end  product,  not  the  p'ocess.  This  is  a  key  point  because 
workers  will  concentrate  on  meeting  whatever  it  is  on  which  they  are  being  evaluated, 
usually  something  quantifiable  such  as  a  quota.  Because  the  emphasis  is  on  the  end  result, 
they  will  not  devote  time  to  improve  the  process.  Deming  submitted  that  employees  fall 
within  a  distribution  whereby  all  but  a  small  number  will  lie  within  the  control  limits,  or 
within  the  system  as  he  termed  it.  He  stated  that  trying  to  rank  order  those  performing 
within  the  system  is  impossible  because  the  performance  of  anyone  is  a  combination  of 
many  forces  of  the  system  itself  (Deming,  1986;  101-1 20) 

In  the  same  fashion  as  Deming,  Scholtes  offers  seven  arguments  against  the 
performance  appraisal  (Scholtes,  1993:354).  First,  the  performance  appraisal  undermines 
teamwork  because  its  focus  is  most  likely  on  individual  performance.  Second,  the 
appraisal  encourages  employees  to  work  around  the  system  for  personal  gain  rather  than 
to  improve  the  system  for  the  benefit  of  the  organization.  He  contends  that  the  fallacy 
behind  the  appraisal  is  the  belief  that  the  rater  can  distinguish  the  individual’s  contribution 
from  all  of  the  other  influences  within  the  system  This  is  rarely  possible,  which  casts 
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doubt  on  the  fundamental  accuracy  of  the  appraisal  itself  Scholtes  maintains  that 
“distorting  the  numbers,  a  form  of  creative  accounting  aimed  at  looking  good  rather  than 
doing  well  is  rampant  in  American  business”  (Scholtes,  1993:354)  Third,  appraisals  use 
measurement  criteria  that  are  unreliable  and  inconsistent.  This  is  primarily  due  to  the 
presence  of  the  rater’s  subjectivity  and  lack  of  rater  training.  Fourth,  appraisals 
encourage  an  approach  to  problem-solving  that  attempts  to  point  the  finger  at  those  in  the 
system.  Scholtes  states  that  the  major  difference  in  problem-solving  between  Japanese  and 
American  managers  is  that  Japanese  managers  ask,  why?  while  their  American 
counterparts  ask,  who?  Fifth,  appraisals  tend  to  establish  safe  goals  in  an  organization. 
Workers  will  be  measured  against  goals  that  are  easy  to  meet  and  they  will  create  an 
illusion  of  challenge  around  those  easy  targets.  Consequently,  if  everyone  in  the 
organization  is  striving  toward  these  easy  targets,  the  organization  is  bound  by  a  ceiling  of 
mediocrity  Sixth,  appraisals  create  cynics  and  wasted  resources.  Managers’  expectations 
can  have  a  tremendous  impact  on  worker  pjerformance.  A  manager’s  perception  of  a 
subordinate’s  performance  can  result  in  a  self-fiiifilling  prophecy,  a  situation  in  which  a 
person  will  behave  or  perform  to  a  level  which  the  rater  expects,  a  perception  later 
communicated  via  the  appraisal.  Additionally,  when  the  completed  appraisal  is 
inconsistent  with  how  a  worker  feels  he  or  she  performed,  responses  such  as  fiustration, 
lower  job  satisfaction,  or  mistrust  can  develop  (Moss,  1989:59).  Seventh,  appraisals  are 
used  to  perform  several  managerial  functions  in  one,  yet  are  inadequate  to  perform  any 
one  of  them.  Scholtes  lists  uses  such  as  promotion  identification,  feedback,  a  directional 
tool,  career  development,  and  training  need  verification  (1993:353-359)  Moen  submits 
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that  tor  many  organizations,  the  unstated  purpc  ;  of  the  performance  appraisal  is  control 
of  the  employees.  This  reflects  a  management  style  that  discourages  intrinsic  motivation, 
self-esteem,  and  teamwork  (Moen,  1989:62).  Finally,  Bowman  states  that  performance 
appraisals  are  widely-used  because  “they  are  the  easy  way  out:  they  do  not  require  anyone 
to  address  the  problems  employees  face”  (Bowman,  1 994: 130).  This  statement  is 
consistent  with  Deming’s  argument  that  appraisals  tend  to  focus  the  attention  on  the 
employees  when  it  comes  to  problem  solving  rather  that  focusing  on  systemic  factors  that 
could  be  the  problem.  Bowman  goes  on  to  say,  “The  process  should  be  diagnosed,  not 
the  employees  blamed”  (Bowman,  1994: 130). 

Possible  Solutions.  If  the  performance  appraisal  is  in  conflict  with  the  foundation  and 
principles  of  Total  Quality  Management  as  this  literature  suggests,  how  then  can  an 
organization’s  performance  (and  those  within  it)  be  evaluated? 

According  to  Deming,  the  first  and  most  critical  step  is  to  provide  education  in 
leadership.  He  maintains  that  leadership  must  move  fi'om  the  performance  appraisal 
system  to  an  appraisal  of  the  performance  of  the  system.  Every  person  should  know  the 
purpose  of  the  organization  (the  system)  and  how  his  or  her  role  ties  into  that  purpose. 
Thus,  once  people  know  their  impact  on  other  elements  of  the  system,  they  can  take 
control  of  the  processes  and  improve  those  that  are  inhibiting  the  system’s  performance, 
rather  than  have  the  previously  mentioned  carrot-and-stick  controls  that  are  externally 
imposed  upon  them.  Deming  also  suggested  that  organizations  more  carefully  select 
people  in  the  first  place  If  this  were  to  occur,  the  performance  appraisal  would  not  be 
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needed  to  determine  if  an  employee  was  fit  to  remain  employed,  which  would  enable  the 
supervisor  to  assume  more  of  a  developmental,  coaching  role.  (Deming,  I9S6: 101 ) 

So,  if  performance  appraisals  don’t  exist  within  an  organization,  how  can  feedbaclc, 
promotion,  compensation  and  development  be  managed?  Scholtes  suggests  the  following 
four  guidelines.  First,  employee  feedback  is  to  improve,  not  control,  and  should  be 
ongoing  and  timely.  Feedback  sources  can  include  peers,  internal  and  external  customers 
and  suppliers  as  well  as  supervisors.  Second,  compensation  (including  bonuses)  should 
not  be  based  on  Actors  beyond  an  employee’s  ability  to  influence,  such  as  quotas.  It 
should  be  based  on  the  accumulation  of  skills  and  responsibilities,  experience,  seniority 
and  the  organization’s  success  (such  as  profit  sharing)  Third,  candidates  for  promotion 
should  be  identified  based  on  the  requirements  of  the  higher  position  and  how  an 
individual’s  c^abilities  and  training  match  those  requirenrents.  Fourth,  training  is  a 
critical  part  of  TQM.  Like  feedback,  training  is  an  ongoing  process  and  both  managers 
and  employees  imist  sibscribe  to  this  philosophy  if  TQM  is  to  be  fully  implemented 
(Sholtes,  1993:359). 

Scholtes  ofiFers  two  alternatives  to  performance  appraisals,  both  of  which  are 

disconcerting  to  managers  because  they  are  radical  changes  to  traditional  Western 

management  practices:  l)Managersmustchangetheway  they  think 

Until  managers  let  go  of  their  obsession  with  the  iiulividual  worker  and 
understand  the  importance  of  systems  and  processes,  they  will  not  enter  the 
quality  era.  Without  this  change  in  mindset,  managers  will  continue  to  look  for 
alternatives  that  are  no  different  than  the  ones  they  are  trying  to  replace. 

(Scholtes,  1993  360) 
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2)  Just  slop  doing  it  He  says  that  a  way  to  develop  alternatives  to  the  appraisal  is  by 
debundling.  He  equates  an  all-in-one  performance  appraisal  to  a  fragile  cart  carrying  too 
far  a  heavy  load.  Managers  must  remove  “each  piece  of  baggage  and  build  for  each,  a 
separate  vehicle  designed  specifically  for  that  function”  (Scholtes,  1993  :360-361). 

Daley  offers  another  alternative  called  a  developmental  appraisal.  His  reasons  for  the 
need  to  transform  the  existing  appraisal  process  coincide  with  those  previously  discussed. 
He  states  that  most  concerns  are  that  the  performance  appraisal  be  developed  from 
judgmental  processes  Developmental  appraisals,  however,  are  not  susceptible  to  this  and 
may  actually  enhance  the  TQM  process.  This  approach  is  intended  to  focus  on  the 
training  and  skills  essential  to  be  successful  in  TQM.  Developmental  appraisals  take  an 
employee’s  basic  competence  for  granted— they  are  not  instruments  to  determine  those 
who  must  be  weeded  out.  As  a  result,  this  places  critical  emphasis  oti  the  recruiting 
function  to  ensure  that  the  right  people  are  hired  in  the  first  place  since  evaluating 
employee  competence  is  not  a  function  of  the  developmental  appraisal  From  there, 
developmental  appraisals  focus  on  developing  the  employee  and  adding  to  the  employee 
package  (Daley,  1992:46).  This  appraisal  produces  two  main  results.  First,  the  employee 
is  intimately  involved  with  his  or  her  own  professional  development.  Second,  this  can 
have  a  positive  psychological  and  intrinsically  motivational  effect  on  the  employee. 
Barriers  between  employees  and  management  arc  broken  by  the  process  of  ongoing 
feedback  which  is  centered  around  the  employee’s  development.  This  facilitates  a  feeling 
of  mutual  trust  and  eliminates  the  intimidation  of  win-lose,  carrot-and-stick  appraisal 
processes  (Daley,  1992:45) 
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Table  4  is  a  comparison  of  approaches  to  the  performance  appraisal  under  the 
traditional  management  versus  the  quality  management  environments  (Bowman, 


1994: 132)  The  idea  conveyed  in  this  table  is  that  organizations  must  make  their  appraisal 
cvstem'"  '<^flect  their  guiding  values.  If  an  organization  wants  to  take  on  the  TQ  approach 
1' .  ihg  business,  it  needs  to  have  an  appraisal  system  that  will  reinforce  the  ideals  behind 
i-  aving  an  appraisal  system  that  reflects  the  criteria  in  the  Quality  Management 
column  should  facilitate  the  organization’s  emergence  in  the  TQ  environment.  Bowman 
maintains  that  if  organizations  will  not  do  away  with  appraisals,  they  should  at  least 
change  they  way  they  are  designed  and  administered.  If  organizations  are  going  to  use 
appraisals,  they  must  be  designed  and  administered  in  a  way  that  encourt^es  the  behaviors 
that  are  consistent  with  TQM. 


Table  4,  Comparison  of  Performance  Appraisal  Approaches 


Criteria 

Traditional  Management 

Quality  Management 

Guiding  value 

Attribution  to  individual 

Attribution  to  the  system 

Information  basis 

Individual  behavior; 
conformance  to  the  system 

Work  group  participation; 
continuous  improvement  of 
system 

Rating  scale 

Five  or  more  scale 

Three  scale  categories 

Primary  goals 

Control;  documentation 

Development;  solving 
problems 

Supervisory  role 

Supervisor  as  judge 

Leadership  practices 

Directional;  evaluative 

Facilitative,  coaching 

Appraisal  firequency 

Occasional 

Frequent 

High 

Low 

Reward  practices 

Individual  orientation 

Group  orientation 
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Douglas  McGregor  addresses  the  issue  from  the  manager’s  point  of  view.  He  submits 
that  the  main  problem  with  the  performance  appraisals  is  the  reluctance  of  managers  in 
administering  them.  He  suggests  a  new  approach  of  analysis  versus  appraisal.  The  main 
difference  of  this  approach  is  that  it  rests  on  the  belief  that  the  employee  knows  more  than 
anyone  else  about  his  or  her  own  capabilities,  strengths  and  weaknesses.  This  lends  to  the 
application  of  mutually  derived  appraisal  measure.  He  adds  that  ratings,  aptitude,  and  the 
superior’s  necessarily  limited  knowledge  of  an  employee’s  performance  yields,  at  best,  an 
in^)erfect  picture.  Therefore,  the  proper  role  of  the  supervisor  under  this  approach  is  one 
of  a  coach-helping  his  or  her  subordinate,  and  “is  the  role  the  employee  wants  and  one 
that  the  manager  prefers”  (McGregor,  1 972 ;  1 34- 1 37) . 

Barriers  to  EJiminating  Performance  Appraisals.  A  major  barrier  to  eliminating 
performance  appraisals  is  that  appraisals  are  an  established  tool  in  Western  management. 
Management,  in  general,  likes  appraisals  because  they  are  an  easy  way  out.  Appraisals  do 
not  require  managers  to  identily  and  solve  the  problems  employees  face.  Appraisals  are 
based  on  the  assumption  that  problems  relating  to  performance  can  be  attributed  to  the 
individual.  Once  managers  complete  the  appraisal,  their  jobs  are  done  and  the 
responsibility  for  improving  quality  and  performance  is  placed  on  the  subordinate. 
(Bowman,  1994:129-130) 

Bowman  identified  three  other  reasons  why  it  is  difficult  to  do  away  with  the  appraisal. 
First,  in  traditional  management  circles,  it  is  assumed  that  everyone  cannot  meet  standards, 
therefore,  ratings  need  to  be  placed  on  subordinates  to  justify  the  allocation  of  scarce 
resources,  namely  raises  and  promotions.  If  only  a  few  can  receive  raises  or  promotions. 


there  must  be  a  system  in  place  to  justify  the  reasons  behind  those  that  receive  them. 
Second,  appraisals  support  the  view  that  it  is  necessary  to  keep  subordinates  under 
control  Since  management  has  that  control,  they  are  reluctant  to  relinquish  it.  Third, 
appraisals  induce  fear  in  the  subordinate  and  stifle  open  communication  between  manager 
and  subordinate.  This  results  in  a  lack  of  suggestions  or  alternatives,  which  further 
cements  management’s  belief  that  the  appraisal  is  the  best  way  to  control  what  is  going  on. 

Summary 

The  above  suggestions  for  developing  alternative  methods  for  fostering  a  TQM 
environment  serve  as  a  starting  point  to  mant^ers.  The  main  challenge  for  managers  is  to 
release  the  old  paradigms  of  American  management  and  develop  TQ  processes  for  their 
organisations.  Because  so  many  organizations  have  embraced  the  TQM  philosophy,  there 
is  great  potential  for  an  organization  to  develop  effective  TQ  approaches  by  learning  what 
other  organizations  are  doing.  An  effective  tool  to  use  for  this  purpose  is  called 
benchmarking.  Benchmarking  and  its  applications  as  it  pertains  to  this  research  are 
discussed  in  Chapter  III,  Methodology  Additionally,  Chapter  3  discusses  the  Delphi 
method  and  the  specific  research  design  of  this  study 


m:  Methodolo^ 


Introduction 

As  the  literature  review  suggests,  there  are  conflicting  opinions  regarding  the 
appropriate  use  of  performance  appraisals  in  organizations.  Some  believe  performance 
^praisals  enhance  the  performance  of  the  organization  while  others,  such  as  Deming, 
believe  performance  appraisals  hinder  long-term  performance.  The  goal  of  this  research  is 
to  describe  or  benchmark  performance  appraisals  in  use  by  companies  using  the  Total 
Quality  philosophy  and  to  gain  consensus  among  experts  in  these  companies  on  the 
important  uses  for  performance  appraisals  under  Total  Quality 

The  purpose  of  this  chapter  is  to  outline  the  design  of  the  Delphi  study  as  it  applies  to 
this  thesis.  First,  we  discuss  the  background  of  benchmarking  and  of  the  Delphi  method 
(including  why  it  was  used  for  this  study),  then  we  discuss  the  specific  research  design 
used  to  answer  the  research  and  investigative  questions 

Bcnchmarkiog 

Because  we  are  attempting  to  describe  performance  appraisal  and  identify  best 
practices  of  performance  appraisals  within  TQ  organizations,  benchmarking  is  a  goal 
which  is  an  appropriate  application  for  the  results  of  this  study.  Benchmarking  is  a 
systematic  and  continuous  measurement  process  of  comparing  an  organization’s  business 
process  against  those  of  business  process  leaders  to  gain  information  which  will  help  the 
organization  take  action  to  .mprove  its  performance  (A?QC,  1993  4)  Simply  stated, 
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benchmarking  is  “the  comparison  of  a  given  business  function  across  companies.  It  is 
designed  to  allow  managers  to  understand  how  their  functional  performance  compares 
with  that  of  other  companies,  particularly  those  that  excel  in  that  fimaion”  (Pryor. 

1989:28)  For  example,  Xerox  used  benchmarking  to  adapt  L.L.  Bean’s  warehousing  and 
distribution  processes  to  its  operation  (Pryor,  1989:28).  Since  the  study  employs  a 
systematic  measurement  process  to  compare  performance  appraisal  practices  in  business 
leading  TQ  organizations,  organizations  can  use  the  information  provided  in  this  study  for 
benchmarks  to  improve  their  TQ  environments.  The  benefits  of  benchmarking  and  the 
appropriate  category  of  benchmarking  to  apply  to  this  data  are  discussed  below. 

Benchmarking’s  Benefits.  There  are  three  very  important  benefits  to  benchmarking. 
First,  the  results  obtained  from  benchmarking  can  motivate  those  within  an  organization  to 
adopt  the  new  process.  This  is  because  there  is  proof  that  it  already  works  somewhere 
else.  Second,  resistance  to  change  may  be  lessened  because  these  ideas  for  improvement 
came  from  another  organization.  Also,  there  is  often  inertia  built  up  in  old  processes, 
which  encourages  people  to  continue  in  their  old  habits  Showing  them  that  different 
methods  are  being  used  successfully  elsewhere  can  provide  inceiinve  to  change  Third, 
benchmarking  enlarges  peoples’  experience  base  and  increases  ledge.  The  more 
alternatives  that  an  organization  explores  the  better  its’  chances  of  hnding  a  conipetitive 
edge  in  the  industry  it  operates  (Evans  and  Lindsay,  1993 : 145). 

Categories  of  Benchmarking.  There  are  three  main  categories  of  benchmarking  The 
first,  strategic  benchmarking,  involves  comparing  different  market  strategies  with  their 
impact  on  market  success  The  second,  operational  benchmarking,  focuses  on  specific 
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tunctionJ^l  operations  such  as  engineering,  manufacturing  or  distribution.  Tiie  third 
category,  business  management  benchmarking,  focuses  on  specific  support  functions  of  an 
organization.  These  can  include  management  information  systems,  marketing  or  human 
resources.  (Pryor,  1989:29-30)  Because  this  study  focuses  on  a  support  function, 
performance  appraisal  in  human  resources,  business  management  benchmarking  is  the 
appropriate  method  to  use  with  the  results  of  this  study.  It  should  be  noted  that  the 
researchers  do  not  use  the  results  of  this  research  effort  to  perform  any  type  of 
benchmarking.  Rather,  the  data  is  presented  in  such  a  fashion  as  to  make  benchmarking 
easier  for  any  organization  that  wishes  to  do  so 

Ilie  Delphi  Technique 

The  Delphi  technique  was  chosen  for  this  study  because  it  is  well  suited  for  eliciting  the 
opinions  of  experts.  It  was  developed  by  the  Rand  Corporation  in  the  1960’s  to  develop 
long-range  technical  forecasting.  Delphi  is  “a  method  to  systei^fitically  collect,  evaluate, 
and  tabulate  independent  opiition  without  group  discussion”  (Tersine  and  Riggs  1976:5 1 ). 
This  technique  replaces  traditional,  face-to-face  group  decision  making  processes  with  an 
iterative  approach  through  the  use  of  several  rounds  of  quesuonnaires.  The  intent  is  to 
eliminate  the  disadvantages  associated  with  group  decision  making  processes.  Since  the 
goal  of  the  study  is  to  systematically  describe  the  perfonnance  appraisal  practices  in 
successful  TQ  environments  and  to  elicit  the  opinion  of  organizational  experts  on  the  use 
of  performance  appraisal  in  these  environments,  the  Delphi  technique  is  appropriate  for 


this  study.  This  disadvantages  of  group  decision  making  and  the  advantages/limitation  of 
the  Delphi  method  are  further  explained  in  the  discuss!  .  that  follows. 

Disadvantages  of  Group  Decision  Making.  Having  a  group  of  experts  in  a  face-to- 
face  decision  making  process  has  three  major  disadvantages.  First,  there  is  a  tendency  for 
a  dominant  individual  (by  'drtue  of  status,  reputation,  or  siiiiply  a  strong  personality)  to 
emerge  and  assume  control  or  leadership  of  the  group  .  This  individual  may  guide  or  steer 
the  group  in  a  direction  which  all  members  may  not  want  to  go.  This  can  stifle  creativity 
and  eliminate  from  consideration  alternatives  that  may  lead  to  other  possible  solutions. 
Second,  if  there  is  more  than  one  good  idea  presented,  the  chosen  solution  is  usually 
obtained  through  compromise  rather  than  consensus.  In  choosing  the  best  alternative, 
members  may  be  persuaded  by  who  suggested  the  idea  rather  than  on  the  merits  of  the 
idea  itself  Third,  the  interpersonal  dynamics  of  face-to-face  group  decision  making  may 
stifre  the  deveLpment  of  alternatives  because  some  members  may  be  hesitant  to  offer  a 
suggestion  because  to  do  so  may  require  him  or  her  to  justify  or  defend  the  proposition. 
(Tersine  and  Riggs,  1976:51) 

Advantages  of  Delphi.  The  Delphi  technique  has  several  advantages  over  traditional, 
group  decision  proces.ses. 

Anonymity.  The  identities  of  the  participants  are  not  revealed.  This  enables  one  to 
change  his  or  her  opinion  without  fear  that  others  view  him  or  her  as  indecisive  One  can 
islso  object  or  disagree  v.athout  the  fear  of  being  judged  by  the  others.  Additionally, 
anonymity  reduces  the  haio  effect,  a  situation  in  which  the  opinion  of  a  dominating 
participant  is  given  more  weight  or  credibility  by  the  others.  Finally,  there  is  less 
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occurrence  of  the  bandwagon  effect,  a  situation  in  which  there  is  a  tendency  to  simply  go 
along  with  the  group.  This  is  because  a  participant  does  not  know  what  the  other 
participants’  opinions  are  at  the  time  he  or  she  is  giving  a  response.  (Tersine  and  Riggs, 
1976:51) 

Less  Confrontational  Because  a  participant’s  views  are  isolated  from  the  rest, 
confrontation  is  eliminated.  Only  after  all  responses  in  a  round  of  questions  have  been 
tabulated  are  the  participants  furnished  with  limited  feedback  on  the  responses.  This 
process  is  continued  on  an  iterative  basis  over  several  rounds  until  consensus  or 
convergence  is  reached. 

Many  Uses  The  Delphi  technique  is  a  tool  which  is  well-suited  for  uses  in  both  the 
private  and  public  sectors.  Although  the  Delphi  method  was  originally  used  for  long- 
range  technical  forecasting,  Delphi  has  been  used  in  many  disciplines  including  the  health 
care,  retail,  computer,  and  engineering  fields  as  well  as  in  government  sectors  such  as 
education,  municipalities  and  the  military. 

Limitations  of  Delphi  There  are  several  limitations  to  the  Delphi  technique.  First, 
selection  of  the  particinants  must  yield  a  panel  that  is  both  qualified  and  motivated.  The 
fundamental  principle  of  Delphi  is  to  solicit  the  responses  of  knowledgeable  individuals 
Addit’^’ially,  the  participants  must  be  committed  to  being  a  part  of  the  study  until  its  end. 
Because  the  Delphi  is  an  iterative  process,  the  study  can  last  for  months.  All  participants 
must  be  available  for  the  duration.  Second,  Delphi  is  not  well  suited  for  routine  decision 
making.  Because  Delphi  is  very  time  consuming,  its  use  should  be  limited  to  unstructured 
problem  solving.  Third,  since  Delphi  is  not  conducted  in  a  face-to-face  forum,  geographic 
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dispersion  may  be  a  problem.  Mailings,  facsimiles  (Fax’s)  can  be  misrouted  or  lost, 
resulting  in  time.  Finally,  since  time  deadlines  for  Delphi’s  can  extend  months  into  the 
future,  the  priorities  placed  on  them  may  be  low  (Tersine  and  Riggs,  1976:56).  Selection 
of  qualified  participants  is  discussed  later  in  this  chapter. 

Research  and  Investigative  Questions 

This  study  attempts  to:  1)  identify  the  performance  appraisal  techniques  in  use  by  the 
“best”  TQ  companies  and  2)  elicit  the  opinions  of  the  experts  in  these  companies  on  what 
are  the  most  important  uses  of  performance  appraisal  in  a  TQ  environment.  As  stated  in 
Chapter  I,  the  researchers’  consider  winners  of  the  Malcolm  Baldrige  National  Quality 
Award  to  be  among  the  best  TQ  organizations,  due  to  the  rigorous  requirements  that  must 
be  met  in  order  to  win  the  award.  The  objective  of  this  study  is  to  identify  how  successful 
organizations  bridge  the  gap  between  the  traditional  performance  appraisal  practices  and 
appraisal  practices  that  are  tailored  to  the  Total  Quality  philosophy.  To  answer  this 
research  question,  several  investigative  questions  must  be  ansv/ered  They  are  outlined 
below: 

1 .  What  are  the  elements  and  uses  of  performance  appraisals? 

2.  What  are  the  differences  between  the  traditional  and  TQ-oriented  approaches  to  the 
performance  appraisal? 

3.  Which  characteristics  of  the  performance  appraisal  are  currently  in  use  by  leading  TQ 
organizations? 

4  Which  characteristics  and  uses  of  performance  appraisal  are  viewed  as  most  significant 
by  the  organizational  experts? 
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Selected  Organizatioiis 


Because  this  study  attempts  to  describe  the  performance  appraisal  practices  of 
companies  successftU  in  the  TQ  environment,  one  of  the  key  selection  criteria  is  that  they 
have  demonstrated  TQ  success.  Since  the  Malcolm  Baldrige  Award  is  a  nationally 
recognized  award  for  quality  and  has  a  rigorous  selection  process  which  includes 
personnel  and  human  resoi'.'ces  it  was  used  as  the  selection  criterion.  All  companies 
winning  the  award  in  the  past  eight  years  were  contacted  and  Ten  companies  agreed  to 
participate  in  this  study.  The  companies  were  from  the  transportation,  marketing,  service 
and  manufecturing  sectors.  Geographic  distribution  varied  from  operating  within  one 
community  to  multinational  operations.  The  size  of  the  companies  varied  from  450 
employees  to  over  100,000  employees.  One  company  contained  both  union  and  non¬ 
union  employees  while  the  other  nine  companies  were  comprised  solely  of  non-union 
employees.  Table  5,  below,  provides  a  description  of  the  characteristics  discussed  above. 
It  should  be  noted  that  tiie  expert  from  organization  F  was  not  able  to  participate  after 
round  one  due  to  relocation;  therefore,  organization  F  is  only  included  in  round  one 
responses. 


Table  5,  Organizational  Characteristics 


Geographic  Dist 

Size  (employees) 

Business  focus 

A 

Multi-state 

■'000 

Manufacturing 

B 

Multi-state 

3000 

Service 

C 

Multi-state 

450 

Sales  and  Marketing 

D 

Multi-national 

100,000 

E 

Multi-naUonal 

8000 

Manufacturing 

F 

Multi-national 

105.000 

Manufacturing 

G 

One  Site 

2600 

Manufacturing 

H 

Multi-National 

85.000 

Setvice/Manufaauring 

1 

Multi-national 

96,000 

Service/Manufacturing 

J 

Multinational 

850 

Manufacturing 

Selection  of  Experts 

The  use  of  the  Delphi  technique  requires  soliciting  responses  from  experts.  The 
experts  for  this  study  are  all  human  resource  or  quality  improvement  professionals 
employed  by  each  of  the  ten  Malcolm  Baidrige  National  Quality  Award  winning 
companies.  Personnel  and  quality  professionals  were  chosen  based  on  their  familiarity 
with  both  performance  appraisal  systems  and  TQ.  The  participants’  mean  years  of 
personnel  and  performance  appraisal  experience  is  13  years,  with  the  range  being  2.5-23 
years.  TQ  experience  for  all  participants  was  based  on  years  with  the  organization  and  the 
organizations’  experience  with  TQ.  The  range  was  from  2.5  to  12  years. 

Development  of  Data  Gathering  Instruments 

Respondents  were  sent,  via  electronic  facsimile  transmission  (FAX),  a  series  of  three 
questionnaires  over  a  three-month  period.  The  primary  objectives  in  developing  the 
questionnaires  were  to  1)  identify  the  extent  to  which  the  organizations  used  traditional 
performance  appraisal  practices;  2)  identify  the  extent  to  which  the  companiet  employed 
TQ  philosophies  in  their  appraisals;  3)  identify  what  the  respondents  viewed  to  be  the  key 
elements  of  the  performance  appraisal  in  a  TQ  environment.  The  first  two  objectives 
required  the  collection  of  responses  to  both  open-ended  questions  and  responses  to 
questions  based  on  a  five-point  Likert  scale  ranging  from  “strongly  disagree’’  to  “strongly 
agree’’.  Objective  three  required  data  that  can  be  used  to  weight  the  elements  based  on 
their  relative  importance  to  the  experts’  organizations  and  personal  opinions.  Except  for 
questionnaire  1  and  the  Likert  Scale  questions  in  questionnaire  2  (these  were  developed 
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from  the  literature  review)  the  questionnaires  were  developed  based  on  responses  obtained 
in  previous  rounds  of  questions.  Each  iteration  was  designed  to  further  define  the  use  of 
performance  appraisals  in  a  TQ  environment.  The  questionnaires  for  rounds  1,  2,  and  3 
are  discussed  below  and  are  included  as  Appendices  C,  D  and  E,  respectively. 

Round  One  Questionnaire  The  first  questionnaire  sent  to  the  respondents  attempted 
to  ascertain  the  extent  to  which  various  traditional  and  TQ-oriented  performance  appraisal 
techniques  exist  in  these  organizations.  The  criteria  used  to  measure  this  variable  were 
developed  based  on  an  exhaustive  review  of  the  literature  on  both  performance  appraisals 
and  the  TQ  philosophy.  Respondents  were  asked  to  evaluate  characteristics  of  their 
performance  appraisal  processes  against  characteristics  of  traditional  performance 
^praisals  using  a  standard  1*S  Likert  measurement  scale,  with  1  representing  strongly 
disagree  or  non-existem  and  5  representing  strongly  agree  or  existent.  The  scaled 
questions  asked  whether  respondents  agreed  that  various  characteristics  of  traditional 
performance  appraisal  existed  within  their  organizations.  The  charaaeristics  were 
selected  based  on  what  the  researchers  identified  as  consistent  themes  throughout  the 
literature  relating  to  performance  appraisal  in  other  than  TQ  environments.  These 
included  evaluation  criteria,  uses,  formats,  compensation  policies  and  so  on. 

The  first  questionnaire  also  asked  numerous  open-ended  questions  regarding  the 
characteristics  of  the  respondents  performance  appraisal  processes  and  the  total  quality 
initiatives  that  the  organizations  are  using.  These  were  focused  on  getting  an  initial 
impression  of  various  aspea  of  the  organizations  performance  appraisal  in  the  TQ 
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environment.  The  areas  questioned  included  process  improvement,  empowerment,  team 
performance. 

Round  Two  Questionnaire  The  second  questionnaire  included  a  second  round  of 
questions  and  summary  information  of  the  participants’  responses  to  the  first  round  of 
questions.  Since  the  response  to  the  scaled  questions  in  round  1  indicated  a  high  degree  of 
agreement  on  the  use  of  tradttionai  practices  within  the  appraisal,  a  set  of  Likert  scaled 
questions  was  included  in  round  two  to  measure  the  extent  to  which  TQ  characteristics 
were  being  used  in  the  performance  appraisals.  As  in  round  one  the  questions  were 
developed  based  on  what  the  researcher  deemed  as  consistent  themes  throughout  the 
literature  on  performance  appraisal  under  TQ.  These  characteristics  included  focus, 
evaluation  criteria,  goals.  Also,  because  of  the  high  degree  of  consensus  on  round  one, 
the  respondents  were  requested  to  prioritize  the  vaiious  techniques/practices  that  were 
identified  in  round  one  in  terms  of  their  significance  to  TQ  practices  The  content  and 
uses  of  the  appraisals  were  broken  down  into  various  dements  and  became  the  basis  for 
the  third  round  of  questions. 

Round  Three  Questionnaire  Third  round  weighted  responses  were  used  to  develop 
a  scale  of  performance  uses  the  experts  judged  most  important  for  companies  TQ 
environments.  Additionally  a  number  of  scaled  responses  were  used  to  clarify  responses 
from  previous  rounds. 
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Testing  the  ilatn  Collection  Instruments 

Prior  to  sending  out  each  questionnaire  it  was  presented  to  members  of  the  faculty  to 
evaluate  the  clarity  and  focus  of  the  questions.  Content  validity  was  also  assessed  via 
faculty  review  as  well  as  a  review  of  prior  research.  Additionally,  the  content  of  the  first 
round  instrument  was  validated  by  a  human  resource  professional  of  a  local  company 

Reliability  was  assessed  by  measuring  the  internal  consistency  of  the  various  scaled 
responses.  The  measure  used  to  assess  internal  consistency  was  Cronbach’s  coefficient 
alpha.  Reliability  is  a  measure  of  internal  consistency  fi’om  one  set  of  measurements  to 
another.  For  example,  two  sets  of  measurement  on  the  same  variable  for  the  same 
individual  may  not  have  exactly  the  same  value,  however,  repeated  measurements  of  a 
series  of  individuals  will  show  some  consistency.  Cronbach’s  coefficient  alpha  can  be  used 
to  estimate  the  reliability  coefficient  (SAS,  1985.212).  In  other  words,  Cronbach’s  alpha 
was  used  measure  tlie  extent  to  which  the  scaled  questions  in  round  one  measure  the 
existence  of  traditional  appraisal  characteristics  In  round  two  Cronbach’s  Alpha  was  used 
to  measure  the  extent  to  which  the  scaled  questions  in  round  two  measured  the  existence 
of  TQ  related  characteristics.  The  results  of  this  analysis  of  the  traditional  and  TQ 
constructs  are  contained  in  Tables  6  and  7,  on  the  following  pages. 

The  overall  Cronbach  CoeflScient  Alpha  for  the  traditional  appraisal  variables  in  the  test 
is  .84052.  Tliis  alpha  is  significant  at  the  05  level.  It  can  be  concluded  at  the  .05  level 
that  the  items  listed  provide  measure  of  the  existence  of  the  criteria  traditional  appraisal. 

In  other  words,  the  scaled  questions  appear  to  be  a  reliable  (internally  consistent)  measure 
for  the  construct  traditional  appraisal.  The  Cronbach  coefficient  alpha  for  the  TQ  related 
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questions  is  .8965.  Again  this  is  significant  at  the  05  level  and  it  would  appear  the 
question  provide  a  reliable  measure  for  the  criteria  TQ  appraisal. 


Table  6,  Cronbach’s  Alpha  for  Traditional  Appraisal  Constructs 

Cronbach  Coefficient  Alpha 

for  RAW  variables  ;  0.840522 
for  STANDARDIZED  variables:  0.845594 


Question:  ALPHA 


Ql .  Measurable  Objectives  are  established  for  each  employee 

0.824345 

Q2.  The  appraisal  process  is  linked  to  employee  development  programs 

0.834386 

Q3.  Our  organization  has  established  education  programs  to  effectively  train 
raters  about  the  appraisal  processes 

0.849001 

Q4.  Our  organization  has  established  education  programs  to  effectively  train 
raters  about  tlte  appraisal  purpose* 

0.829785 

Q5 .  Our  appraisal  process  is  linked  to  reward  decisions 

0.825553 

Q6.  Our  appraisal  process  is  linked  to  promotion  decisions 

0,824242 

Q7.  Our  appraisal  process  is  linked  to  layoff  decisions 

0.813223 

Q8.  t.}jur  appraisal  process  is  linked  to  dismissal  decisions 

0.832296 

09.  Our  appraisal  process  is  used  as  a  tool  for  improving  performance 

0.835556 

QlO.  Our  appraisal  is  used  as  a  tool  for  improving  productivity 

0.839220 

Qll.  Our  appraisal  is  used  as  a  tool  for  facilitating  employee  training 

0.835332 

012.  Our  appraisal  Is  vised  as  a  tool  for  facilitating  employee  development 

0.826584 

013.  0»ir  appraisal  is  used  as  a  tool  for  identifying  promotable  employees 

0.811069 

015.  Our  appraisal  measures  mdividual  performance 

0.839991 

Ql7.  Ovir  appraisal  is  designed  to  evaluate  performance  in  terms  of 
contribution  to  the  end  product 

0.838389 

Q18.  The  performance  criteria  in  the  appraisal  is  tied  into  and  consistent  with 
overall  goals  of  the  organization 

0.838114 

Q20.  Appraisal  ratings  are  specifically  tied  to  performance  based  on  written 
job  descriptions 

0.842105 
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Table  7,  Cronbach’s  Alpha  for  TQ  Appraisal  Constructs 
Correlation  Analysis 
Cronbach  Coefficient  Alpha 


for  RAW  variables  :  0.869565 

for  STANDARDIZED  variables:  0  881692 

Question: 

ALPHA 

m .  Our  organization  rewards/compensates  based  on  team  perfonnance 

.858624 

IT2.  Team  perfonnance  is  a  criterion  ^^ch  is  rated  on  our  performance 
appraisal 

0.870977 

ITS ,  Continuous  empiovee  unprovement  is  the  goal  of  our  appraisal 

0.865457 

rr4.  Promoting  continuous  process  improvement  is  a  .goal  of  our  appraisal 

0.871156 

ITS.  Our  organization  rewards  individuals  based  on  individual  appraisal 
ratings 

0.871875 

rT6.  Our  organization  focuses  on  the  perfonnance  of  the  system  vs 
perfonnance  of  the  individual 

0.867976 

' 

IT?.  Our  appraisal  is  used  mainly  as  a  source  of  feedback  to  the  employee  to 
improve  employee  performance 

0.854084 

ITS .  Compensation  is  based  on  an  accumulation  of  skills 

0.865966 

rr9.  Compensatioa  is  based  on  an  accumulation  of  resptmsibihties 

0.859383 

mo.  Ccmipensation  is  based  on  the  organizarion/department  success 

0.851423 

m  1 .  Our  appraisal  takes  an  employee’s  basic  competence  for  granted 

0.877316 

m2.  Our  appraisal  could  be  characterized  as  a  coaching  instrun^t 

0.868864 

m3.  The  organization  compensates  based  on  team  performance 

0.857464 

m4.  The  organization  rewards  based  on  team  performance 

0.861436 

ms.  Performance  goals  are  established  by  teams 

0.851605 

m6.  Participation  in  process  improvement  teams  is  an  important  element  in 
the  organization's  appraisal 

0.859228 

m?.  The  organization's  appraisal  is  used  for  fiiture-oriented  purposes 

0.860764 

ms.  The  organizaticm's  appraisal  is  used  to  reflect  on  past  performance 

0.884954 

m9.  The  ratee  is  encouraged  to  review  the  completed  appraisal  before  it 
becomes  a  matter  of  record 

0.863677 

IT20.  Throughout  the  organization,  raters  interpret  the  rating  procedures 
imiformly 

0.868814 

0.856482 

IT22.  My  organizations  appraisal  philosophy  is  representative  of  Demings  third 
“deadly  sin” 

0.872815 

IT23.  Our  appraisal  is  conductive  to  fostering  empiovee  empowerment 

0.867650 

IT24.  Our  appraisal  is  designed  to  evaluate  performance  in  ternis  of 
contribution  to  process  improvement 

0.855592 

In  addition  to  these  measures,  the  Delphi  technique  has  several  inherent  characteristics 
which  improve  its  reliability  and  validity'.  “Delphi  technique  is  a  general  methodology  for 
achieving  a  reliable  consensus  of  opinion  from  a  group  of  experts.  ”  (Tersine  and  Riggs. 
1976:56)  The  Delphi  technique  employees  three  features  which  increase  validity  and 
reliability.  These  three  features  are;  1)  anonymity;  2)  controlled  feedback;  and  3) 
statistical  group  response.  Methods  to  ensure  anonymity  include  the  questionnaire.  These 
serve  as  ways  of  reducing  the  impact  of  a  dominant  individual.  The  second  feature, 
controlled  feedback,  involves  conducting  the  exercise  in  a  sequence  of  rounds  between 
which  a  summary  of  the  results  of  the  previous  round  are  communicated  to  the 

participants  This  is  a  device  used  to  facilitate  convergence  of  responses  The  third 

* 

feature,  use  of  statistical  definition  of  the  responses,  is  a  way  of  reducing  group  pressure 
for  conformity.  (Daikey,  1969: 16)  The  Delphi  technique  should  provide  results  that  are 
reproducible  (Daikey  1969: 12). 

Summary 

This  thesis  employs  the  Delphi  tnethod  to  describe  and  benchmark  performance 
appraisal  uses  of  successftil  TQ  organizations.  The  Delphi  approach  is  an  accepted 
research  methodology  for  gaining  consensus  among  experts.  Benchmarking  is  also  an 
accepted  approach  to  identify  “best  practices”  of  organizations.  As  stated  earlier, 
“benchmarking  should  not  be  limited  to  organizations  within  the  same  field  or  industry  In 
Chapter  IV.  the  data  obtained  using  the  Delphi  procedure  is  presented  and  analyzed 
Chapter  V  presents  the  results  of  the  Delphi  study. 
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rV:  Findings  and  Analysu 


Introduction 

This  chapter  is  divided  into  three  main  sections.  The  first  section  analyzes  the  data 
firom  the  three  rounds  (the  data  for  round  one,  two,  and  three  are  presented  in  appendix's 
G,  H,  and  I  respectively).  The  second  section  provides  a  summary  of  the  data  provided  by 
organization  J,  which  does  not  use  performance  appraisal  in  its  organization,  and  therefore 
was  not  able  to  respond  to  our  questionnaires,  but  did  provide  information  on  its 
organizations’  practices.  Finally,  section  three  presents  the  data  firom  this  study  to  answer 
the  four  investigative  questions  that  were  posed  in  Chapter  I.  The  10  organizations  that 
participated  in  the  study  are  referred  to  as  Organizations  A  through  J.  It  is  important  to 
keep  in  mind  Organization  A  has  no  written  performance  appraisal  for  its  non-exempt 
(wage  earning)  employees  as  per  its  union  contract.  The  responses  fi-om  Organization  A 
refer  to  its  appraisal  system  for  its  exempt  (salaried)  employees. 

Analysts  of  Scaled  and  Weighted  Questions 

The  scaled  questions  fi-om  rounds  one,  two.  and  three  were  divided  into  two 
categories.  The  first  category  contains  those  items  designed  to  measure  the  existence  of 
traditional  practices  toward  performance  appraisal.  Although  this  category  may  contain 
constructs  espoused  by  TQ  experts,  the  items  in  this  category  were  expressed  prior  the 
popularization  of  TQ.  The  second  category  contains  constructs  identified  primarily  with 
the  TQ  movement. 
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Traditional  Apprsual  Constructj.  Table  8  below  contains  the  traditional  appraisal 
constructs,  along  with  the  mean  and  standard  deviation  (SD)  for  each  construct.  The 
summed  values  for  all  constructs  are  also  presented.  Figure  1,  on  the  following  page,  is  a 
histogram  of  all  responses  in  the  category  regarding  the  traditional  approaches  to  the 
performance  appraisal.  This  histogram  represents  the  frequency  of  each  rating,  based  on 
the  1  (strongly  disagree)  through  5  (strongly  agree)  Likert  scale.  Since  all  questions  were 
posed  in  sitch  a  manner  that  a  response  of  5  is  always  a  positive  response,  it  is  apparent  by 
looking  at  the  histogram  that  the  overwhelming  number  of  responses  were  either  agree  or 
strongly  agree.  This  indicated  that  these  organizations  still  incorporate  the  traditional 
methods  of  using  the  performance  appraisal. 


Tid)le7,  Traditional  Appraisal  Constructs 


Construa  _  Mean  SD 


0 1 .  Measurable  Objectives  are  est^lisbed  for  each  employee 

1  4.22 

wsm 

4.33  1 

1.32 

Q3.  Our  organization  has  established  education  programs  to  effectively  train  raters 
about  the  appraisal  processes 

3.22 

1.39 

Qi.  Our  organization  has  established  education  programs  to  effectively  train  raters 
about  the  appraisal  purposes 

3.55 

1.23 

i _ 

Q5  Our  appraisal  process  is  linked  to  reward  decisions 

Q6.  Our  appraisal  process  is  linked  to  promotion  decisions 

4.11 

1.16 

Q7.  Our  appraisal  process  is  linked  to  layoff  decisions 

3.66 

1.66 

08.  '^ir  appraisal  process  is  linked  to  dismissal  dedsons 

4,55 

1.33 

09.  Our  appraisal  process  is  used  as  a  tool  for  improvinj;  perfonnance 
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QiO.  Our  appraisal  is  used  as  a  tool  for  improving  producUMCv 

BCTi 

1.01 

Oil.  Our  appraisal  is  used  as  a  toot  for  ^diluting  employee  training 

1.09 

012.  Our  appraisal  is  used  as  a  tool  lot  facilitating  employee  development 

4.44 

1.01 

013.  Our  appraisal  is  used  as  a  tool  for  identifying  promoufale  employees 

2.66 

1.58 

015.  Our  appraisal  measures  individual  oerfomiancc 

4.77 

44 

Q17  Our  appraisal  is  designed  to  evaluate  perfonnance  in  terns  of  contiibuUon  to 
the  end  product 

3.77 

1  39 

1 

018.  The  performance  cntcria  in  the  appraisal  is  tied  into  and  consistent  with  overall 
goals  of  the  organization 

466 

j 

i"' 

Q20.  Appraisal  ratings  are  spectfically  tied  to  performance  based  on  written  job 
descriptions 

3  11 

1 

1 

i  1.32 
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Histogram 


Figure  1,  Histogram  of  Traditional  Appraisal  Constructs. 

TQ  Appraisal  Constructs.  These  constructs  are  primarily  the  product  of  the  second 
round  of  questions.  However,  items  14,  16,  19,  and  33a  from  the  first  round  of  scaled 
questions  were  added  to  the  TQ  appraisal  category.  These  are  presented  on  the  following 
page  as  items  21,  22,  23  and  24  in  Table  8.  This  table  contains  the  TQ  appraisal 
constructs,  along  with  the  mean  and  standard  deviation  (SD)  for  each  construct.  Figure  2 
is  a  histogram  of  all  responses  in  the  TQ  related  appraisal  category.  Notice  there  is  much 
greater  dispersion  in  the  responses  to  the  TQ  related  questions,  possibly  indicating  that  the 
organizations  do  not  use  the  same  methods. 
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TABLE  8,  TQ  Appraisal  constructs 


Construct 


Mean  SD 


m .  Our  organization  rewards/compensates  based  on  team 
performanf'e 

3.5 

1.07 

IT2.  Team  performance  is  a  criterion  which  is  rated  on  our 
performance  appraisal 

3.5 

ns .  Continuous  employee  improvement  is  the  goal  of  our  appraisal 

mm 

1.06 

1X4  Promoting  continuous  process  improvement  is  a  goal  of  our 
appraisal 

3  75 

1.04 

IT5  Our  organization  rewards  individuals  based  on  individual 
appraisal  ratings 

3,62 

1.69 

IT6.  Our  organization  focuses  on  the  performance  of  the  system  vs. 
performance  of  the  individual 

2.62 

_ 1 

.91 

IT7.  Our  appraisal  is  used  mainly  as  a  source  of  feedback  to  the 
employee  to  improve  employee  performance 

’  31 

ITS.  Compensation  is  based  on  an  accumulation  of  skills 

2.75 

1.28 

1X9.  Compensation  is  based  on  an  accumulation  of  responsibilities 

3.75 

1.58 

ITIO.  Compensation  is  based  on  the  organization/depmment  success 

3.25  _ 

1.17 

mi.  Our  appraisal  takes  an  employee’s  basic  competence  for  granted 

2.62 

1.19 

1X12  Our  appraisal  could  be  characterized  as  a  coaching  instrument 

3.62 

1.19 

1X13.  The  organization  compensates  based  on  team  performance 

3.5 

1,20 

1X14.  The  organization  rewards  based  on  team  performance 

4.0 

1.07 

1X15.  Peiformance  goals  are  established  by  teams 

2.87 

1.25 

1X16.  Participation  in  process  improvement  teams  is  an  important 
element  in  the  organization's  appraisal 

3.62 

.92 

1X17.  The  organization’s  appraisal  is  used  for  future-oriented  purposes 

1X18.  The  organization’s  appraisal  is  used  to  reflect  on  past 
performance 

3.12 

1.46 

1X19  The  ratee  is  encouraged  to  review  the  completed  appraisal 
before  it  becomes  a  matter  of  record 

4,0 

1.86 

1X20.  Throughout  the  organization,  raters  interpret  the  rating 
procedures  uniformly 

3.12 

.64 

IT?  1 .  Our  performance  appraisal  measures  group  or  team  performance 

2.62 

1.06 

1X22,  My  organizations  appraisal  philosophy  is  representative  of 
Demings  third  “deadly  sin” 

2.5 

1.41 

IT23 ,  Our  appraisal  is  conductive  to  fostering  employee  empowerment 

3.75 

1,28 

1X24.  Our  appraisal  is  designed  to  evaluate  performance  in  terms  of 
contribution  to  process  improvement 

3.5 

1.20 
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Hiatogram 


Figure  2,  Histogram  of  TQ  appraisal  constructs 


Analysis  of  Weighted  Questions.  The  weighted  questions  from  rounds  two  and 
three  of  the  study  served  two  purposes.  The  first  purpose  was  to  evaluate,  in  order  of 
intportance,  how  these  organizations  actually  used  the  performance  appraisal.  This 
measure  was  strictly  descriptive.  The  second  purpose  was  to  identify  the  experts’ 
(respondents’)  opinions  as  to  the  best  uses  of  the  performance  appraisal  in  the  TQ 
environment. 

The  responses  were  analyzed  using  summary  statistics.  For  the  purpose  of  describing 
the  organizations’  existing  uses  of  the  appraisal,  the  mean  and  range  were  developed  for 
each  construct.  These  are  depicted  in  Table  10. 

For  the  purpose  of  identifying  the  respondents’  vie’.vpoints  on  how  the  appraisal  should 
be  used  in  a  TQ  environment,  the  mean  and  range  were  also  calculated,  however  the 
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variability  of  the  responses  was  also  used  to  determine  the  degree  of  consensus  among  the 
respondents.  For  the  purposes  of  this  study,  any  question  that  had  a  range  of  greater  than 
15  was  determined  not  to  have  gained  consensus  Accordingly,  those  questions  were 
included  in  round  three  in  an  attempt  to  gain  consensus  of  the  experts.  Additionally,  any 
one  use  had  a  mean  of  less  than  8  was  deemed  to  be  an  insignificant  use  of  the  appraisal  in 
a  TQ  environment  and  was  not  included  in  round  three.  Satisfaction  of  both  the  criteria 
were  required  for  inciusion  in  round  three.  The  responses  in  round  three  regarding  the 
experts’  opinions  of  the  best  uses  of  performance  appraisals  in  a  TQ  environment  are 
included  in  Table  1 1  on  the  following  page. 


Table  9,  Round  Two  Weighted  Responses 


Construct 

Oreanization 

Mean/range 

Mean/ range 

Used  for  determining  promotable  employees 

9.375(0-20) 

8.13(0-20) 

Used  for  making  layoff  decisions 

4.375(0-10) 

1.25(0-5) 

Used  tor  making  dismis.sal  decisicms 

6.25(0-15) 

3.75(0-15) 

Used  for  future  development  of  the  ratee 

16.25(5-70) 

32.5(10-70) 

Used  to  categorize  the  ratee  based  on  past  performance 

3.75(0-10) 

0.00(0-0) 

Used  for  making  compensauon  decisions 

18.75(0-50) 

8.75(0-20) 

Used  to  foster  teamwoik 

3.75(0-10) 

8.13(0-15) 

Used  to  fostiT  employee  empowerment 

5.00(0-15) 

10.62(0-20) 

Used  to  evaluate  performance  based  on  objective  criteria 

13.750(0-30) 

15.00^0-40) 

Used  to  evaluate  performance  based  on  subjective  cnteiia 

12.5(0-35) 

6.675(0-15) 

_ 

Used  to  justify  compensation 
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Only  the  five  uses  below  met  the  criteria  to  warrant  their  inclusion  in  the  third  round  of 
questions.  All  other  possible  uses  not  only  met  the  cmerion  for  achieving  consensus,  they 
also  satisfied  the  criterion  for  being  an  insignificant  use  of  the  performance  appraisal  in  a 
TQ  environment. 

The  experts  were  asked  to  reevaluate  the  five  possible  uses  that  are  included  in  Table 
10  below.  Instead  of  having  100  points  to  allocate,  they  had  only  50  points  to  allocate  in 
round  three.  This  enabled  a  more  direct  comparison  of  the  third  round  results  with  the 
second  round  results  since  the  number  of  points  per  question  was  roughly  the  same. 


Table  10,  Round  Three  Weighted  Responses. 

Construct  Response 


The  best  uses  of  the  ^praisal  in  a  TQ 

Environment  are. . . . 

Mean 

Range 

Used  for  detemuning  promotable  employees 

3.75 

0-10 

Used  as  a  tool  for  future  development  of  the  ratee 

20.63 

5-35 

Used  for  making  ccanpoisation  decisitms 

12.5 

5-25 

Used  to  foster  teamwork 

5,31 

0-12.5 

Used  to  foster  employee  empowerment 

6.56 

0-20 

Organization  J  Data 

Organization  J  has  850  employees  throughout  all  operations.  It  is  non-union  and 
dispersed  to  three  sites  located  both  nationally  and  abroad  Organization  J’s  system  is 
based  on  Dr.  W.  Edward  Deming’s  belief  that  most  employees  are  doing  their  very  best  to 
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contribute  to  the  departments’  and  companies’  mission.  Further,  the  company  realizes 
that  any  variation  in  performance  among  employees  may  be  due  to  factors  outside  of  the 
employees  control  and  are  not  necessarily  attributable  to  differences  in  individual 
coir  .cte.  ce.  We  cannot  accurately  measure  performance  due  to  normal  variation  in 
fact,  'r  beyond  the  control  of  the  employees.  The  remainder  of  the  discussion  on 
Organization  J  is  directly  from  information  provided  by  Organization  J’s  expert. 

Organization  J  abolished  its  traditional  performance  appraisal  system  in  1986,  because 
they  believed  it  was;  1)  not  working;  and  2)  did  not  conform  with  Dr.  Deming’s 
philosophy.  In  order  to  take  the  fear  out  of  performance  evaluation,  they  have  no  rating 
system  to  measure  employee  performance.  They  focus  on  employee  development, 
continuous  learning,  and  constant  improvement. 

Organization  J  does  have  an  employee  feedback  mechanism  that  is  a  two-way 
comma’'ication  system  that  takes  place  between  the  manager  and  the  employee.  There  are 
basically  two  outputs;  one  is  a  job  description  that  lists  major  on-going  job  responsibilities, 
and  identifies  the  internal  and  external  customers  who  receive  the  products  or  services  the 
employee  produces;  the  other  output  is  a  Personal  Action  Plan,  which  is  describfed  below 
Job  Descriptions  are  really  statements  of  mutual  expectation  between  the  manager  and  the 
employee.  They  should  be  changed  afrer  a  nugor  change  in  responsibility  but  such 
changes  do  not  necessarily  mean  a  change  in  grade  level  is  required.  This  process  is  used 
for  non-production  employees  only. 

Personal  Action  Planr  are  primarily  the  responsibility  of  the  employee.  The  employee 
and  the  manager  complete  this  part  of  the  process  through  discussion  of  options  that 


would  assist  the  employee  in  further  development  courses/seminars.  Our  intent  is  for 
managers  to  provide  suggestions  to  employees  that  would  help  them  improve  their 
performance  as  part  of  their  overall  development.  The  employee  ultimately  writes  and 
owns  the  plan  that  has  been  negotiated  with  the  manager 

Organization  J  has  not  suffered  from  the  abolition  of  the  performance  appraisal. 
Employees  were  h^py  to  see  the  rating  system  removed  because  it  was  perceived  as  a 
tool  that  induced  fear.  Also,  employees  are  highly  motivated  and  take  ownership  for 
quality. 

For  discipline  problems.  Organization  J  uses  a  progressive  discipline  model,  first 
verbally  communicating  the  concern,  then  documenting  when  improvement  does  not 
occur,  and  lastly  specifying  an  improvement  period  if^  or  when,  failure  to  improve  may 
result  in  termination,  A  zero  increase  at  the  armual  salary  review  may  be  recommended. 
To  quote  Organization  J’s  expert,  “We  have  seen  favorable  results  even  when  the 
problems  have  been  serious.  Our  managers  have  done  a  good  job  of  working  with 
employees  to  improve  performance  rather  than  trying  to  weed  them  out.” 

The  compensation  system  is  divided  into  two  programs.  One  for  non-production 
employees  and  one  for  production  employees.  It  follows  the  theory  that  since  ail 
employees  are  contributing  their  best,  they  will  receive  the  same  dollar  increase  as  their 
peers.  Upon  the  managers’  recommendations,  employees  will  receive  an  annual  wage 
increase  based  on  a  percentage  of  the  midpoint  of  their  grade. 

Organization  J  does  differentiate  performance  to  some  degree.  They  have  an 
exceptional  performer  increase  that  was  intended  for  the  5%  of  the  population  who 
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continuously  exceed  expectations.  Employees  are  recommended  by  their  managers  and 
reviewed  at  the  senior  staff  level  for  approval  of  an  exceptional  performer  increase  at  the 
time  of  our  yearly  salary  forecasting.  Exceptional  performer  increases  are  set  at  50%  to 
100%  of  the  added  to  other  yearly  raises.  According  to  the  Organization  J  expert,  the 
difficulty  has  been  in  defining  the  exact  steps  an  employee  must  take  to  be  designated  as 
exceptional. 

Managers  may  recommend  promotion  fi*om  one  grade  to  another  based  on  increased 
responsibility  or  performance.  Promotional  increases  can  occur  anytime  during  the  year. 
Organization  J  does  not  have  sales  quotas  or  incentives  Executives  are  also  on  straight 
salary  and  have  no  special  perquisites  (perks)  in  benefits  or  compensation  packages. 

Organization  J  says  its  system  works  because  of  constancy  of  purpose.  It  establishes 
four  over-arching  corporate  objectives  each  year  that  drive  their  short-term  plan  The 
short-term  objectives  complement  the  long  range  strategic  plan  and  consequently  there  is 
great  visibility  for  employees  about  what  is  important.  Everyone  is  able  to  identify  his  or 
her  part  in  the  whole,  and  to  have  objectives  that  their  work  group  can  control  through 
their  performance.  They  feel  that  the  system  provides  sufficient  direction  to  employees  on 
what  is  required  for  quality  performance. 

Answers  to  Investigative  Questions 

This  section  discusses  the  results  of  this  study  in  terms  of  how  they  relate  to  the  four 
investigative  questions  that  were  posed  in  Chapter  1. 
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Im  estigative  Question  Number  1:  Based  on  the  available  literature,  what  are  the 


elements  and  uses  of  performance  appraisal?  The  list  of  elemepts  and  uses  of 
performance  appraisal  are  presented  in  detail  in  Chapter  H;  however.  Tables  12  and  13 
below  contain  a  summary  of  the  elements  and  uses  of  performance  appraisal  found  in  the 
literature  review.  These  tables  contain  the  generic  elements  found  in  the  literature  and 
represent  those  elements  and  uses  which  are  not  designed  specifically  for  TQ 
environments.  Table  !  1  depias  the  most  common  uses  (purposes)  of  performance 
^praisal  while  Table  12  represents  the  elements  of  performance  appraisal. 

Table  11,  Common  Puqioses  for  the  Performance  Appraisal  (Moen,  1989:62) 


Purpose 

Percent  of  those  responding 

^  85.6 

Counseling 

65.1 

Training  and  Development 

64.3 

Promotion 

45.3 

Staff  Planning 

43.1 

Retention/Discharge 

30.3 

Validation  of  Selection  Technique 

17.2 

Table  12,  Elements  of  the  Traditional  Appraisal 


Criteria 

Traditional  Management 

Guiding  value 

Attribution  to  individual 

Information  basis 

Individual  behavior,  conformance  to  the 
system 

Rating  scale 

Five  or  more  scale 

Control,  documentation 

Supervisory  role 

Supervisor  as  judge 

Leadership  praaices 

Directional;  evaluative 

!  Occasional 

ISItllSESSSSSSHHMHHHiHI 

High 

Reward  practices  ' 

Individual  orientation 
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The  definition  of  performance  appraisal  is  “a  technique  for  measuring  employee  actions 
against  standards  of  acceptable  performance”  (Plachy  1983:57).  The  five  basic  elements 
of  the  appraisal  are:  1)  measurable  objectives  for  each  position;  2)  making  performance 
appraisal  a  continuous,  year-round  objective;  3)  linking  the  process  to  positive 
development;  4)  educating  appraisers  about  performance  appraisal  purposes  and  praaices 
(Sahl,  1990:55);  and  5)  the  employee  should  be  afforded  the  opportunity  to  review  and,  if 
desired,  appeal  the  performance  rating  (Daley,  1992:40).  Although  there  are  other 
research  efforts  that  identify  other  elements,  most  include  these  five. 

Investigative  Question  Number  Two:  What  are  the  differences  between  traditional 
and  TQ-oriented performance  appraisals'^  The  primary  diflferences  between  traditional 
and  TQ  oriented  performance  appraisals  can  be  identified  by  comparing  the  scaled 
questions  of  round  one  to  the  scaled  questions  in  round  two.  These  questions  were 
developed  based  on  a  review  of  the  literature.  Table  13.  on  the  following  page,  presents  a 
con^)arison  of  the  performance  appraisal  approaches  in  a  traditional  organizational  setting 
versus  a  TQ  environment.  The  primary  differences  between  the  two  approaches  deal  with 
focus.  For  example,  the  traditional  approach  focuses  on  control  and  documentation  as  the 
primary  goals,  while  the  TQ  approach  aims  for  development  and  problem  solving. 
Additionally,  the  TQ  approach  attempts  to  remove  any  elements  which  would  induce  fear 
m  the  individuals  being  rated.  It  focuses  on  the  future  (what  the  employee  can  become) 
not  the  past. 
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Table  13,  Comparison  of  Performance  Appraisal  Approaches  (Bowman,  1994;  132) 


Criteria 

Traditional  Management 

Quality  Management 

Guiding  value 

Attribution  to  individual 

Attribution  to  the  system 

Information  basis 

Individual  behavior; 
conformance  to  the  system 

1 

Work  group 

participation;  continuous 
improvement  of  system 

Five  or  more  scale 

Three  scale  categories 

Primary  goals 

Control,  documentation 

Development;  solving 
problems 

Supervisory  role 

Supervisor  as  judge 

Supervisor  as  coach; 
peers  as  colleagues 

Leadership  practices 

Directional,  evaluative 

Facilitative;  coaching 

Occasional 

Frequent 

High 

Low 

Reward  practices 

Individual  orientation 

Group  orientation 

Investigative  Question  Number  Three:  What  characteristics  of  the  performance 
appraisal  are  currently  in  use  in  the  TQM  environment?  The  answer  to  this  question  lies 
in  the  data  which  was  presented  previously  in  this  chapter.  First,  this  section  addresses  the 
characteristics  identified  in  the  scaled  reposes  to  the  questionnaires  followed  by  the 
weighted  responses  and  open-ended  responses.  The  open-ended  questions  are  divided 
into  traditional,  TQ.  and  other  characteristics  of  the  participant  organizations 

Scaled  Questions.  The  primary  goal  of  the  scaled  questions  used  in  this  research 
was  to  identify  the  respondent  organizations’  degree  of  concurrence  with  traditional  and 
TQ  appraisal  constructs  developed  fi’om  the  literature  review  Figure  3  contains  the  mean 
values  for  the  traditional  appraisal  construas  that  appeared  in  round  one  measured  by  the 
Likert  scales.  There  is  a  high  degree  of  agreement  on  most  of  these  items  The  mean  for 
the  composite  of  these  constructs  is  4.22,  indicating  a  high  degree  of  agreement  on  the 
existence  of  traditional  appraisal  approaches  within  these  organizations  The  only  mean 
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response  bdow  3  (neither  agree  or  disagree)  addresses  the  use  of  appraisal  to  identify 
promotable  employees.  All  other  responses  were  ^ve  three,  which  relates  to  a  high 
degree  of  agreement  on  the  use  these  traditional  appraisal  constructs  by  the  participating 
organizations. 

Figure  4,  on  page  56,  depicts  the  mean  values  for  the  TQ  appraisal  constructs 
measured  by  the  Likert  scales.  There  is  a  less  of  agreement  of  the  existence  of  these 
constructs  currently  in  practice  by  the  participating  organizations.  The  mean  for  the 
composite  of  these  constructs  is  3.4  which  does  not  seem  to  indicate  a  high  degree  of 
agreement  on  the  TQ  appraisal  constructs  among  these  organizations.  The  histogram 
showing  the  frequency  of  each  score  for  each  question  can  be  found  in  Figure  2  on  page 
60.  When  comparing  this  histogram  to  the  histogram  of  the  traditional  appraisal 
constructs,  it  is  apparent  that  the  use  of  the  TQ  constructs  (as  evident  by  a  construct’s 
receiving  a  response  of  5  (strongly  agree)  is  less  conclusive  than  it  is  for  the  traditional 
appraisal  constructs. 


Figure  3,  Average  Response  to  the  Existence  of  Traditional  Appraisal  Constructs 

from  Round  One 

The  questions  below  correspond  to  the  values  on  the  Figure  above. 
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The  questions  below  correspond  to  the  values  in  the  Figure  above 
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Weighted  Questions.  The  weighted  scale  of  interest  here  attempts  to  me;isurp 
how  these  organizations  are  actually  using  performance  appraisals,  later  weighted  M-.if  * 
will  assess  the  experts’  opinions  of  the  best  uses  of  performance  appraisal  in  the  TQ 
environment.  Figure  5,  below,  is  a  graphical  depiction  of  the  mean  weights  for  each  use  of 
the  performance  appraisal. 


ACTUAL  USES  OF  PERFORMANCE  APPRAISAL 
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Figure  5,  Mean  Weights  of  Actu.  1  i  es  of  che  Performance  Appraisal 


These  data  indicate  that  the  use  of  the  appraisal  for  future  development  and 
compensation  decisions  are  the  categories  with  the  two  highest  scores.  Using  the 
appraisal  for  future  development  is  consistent  with  what  the  proponents  of  TQ  supports, 
while  using  the  appraisal  as  a  means  to  determine  compensation  is  not  consistent  with  TQ 
expert  views. 

Open-ended  Responses.  The  open-ended  responses  contains  information  which 
can  be  classified  as  characteristic  of  the  traditional  appraisal  and  the  TQ  appraisal. 
Additionally,  there  are  a  number  of  characteristics  of  the  organizations’  appraisals  which 
are  not  directly  attributable  to  either  category. 

The  charaaeristics  of  the  traditional  appraisal  exhibited  by  the  organizations  include 
the  use  of  standardized  forms,  use  of  forced  scales  (a  numerical  rating),  use  of  certain 
evaluation  criteria,  use  of  the  appraisal  to  hold  individuals  accountable,  and  the  use  of 
appraisals  to  compensate  individuals  for  individual  performance. 

All  organizations,  with  the  exception  of  organizations  I  and  J,  use  standardized 
appraisal  forms  to  evaluate  employee  performance.  According  to  TQ  proponents,  this 
could  inhibit  the  employees  fi'om  developing  their  own  goals  for  development  and 
improvement.  Additionally,  three  of  the  organizations  used  forced  scales  in  their 
performance  appraisal.  Although  TQ  proponents  recommend  the  use  of  a  three  categories 
(below  standard,  quality  performer,  exceptional  performer)  the  organizations  using  rating 
scales  normally  used  a  four  or  more  categories  in  their  scales 

A  number  of  organizations  in  the  study  used  traditional  appraisal  criteria  in  their 
performance  appraisals.  Individual  employee  output,  leadership  ability,  personal  image, 
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and  communication  were  evaluated  by  a  majority  of  the  organizations,  while  absences, 
overtime,  reprimands,  philanthropic  activities,  were  included  in  three  or  more  of  the 
organizations  in  the  study. 

When  asked  about  the  philosophies,  goals,  and  practices  of  their  organizations  (round 
one,  questions  3 1  and  33b),  three  of  the  respondents  expressed  terms  which  indicated  their 
performance  appraisal  was  used  to  access  individual  performance  and  hold  individuals 
accountable  for  performance  and  development.  Additionally,  two  of  the  organizations 
based  pay  increases  solely  on  individual  performance,  while  three  organizations  use 
individual  perfonnance  partially  to  determine  pay  increases. 

Although  most  of  the  respondents  identified  characteristics  of  their  organizations’ 
appraisals  which  were  arguably  traditionally  oriented,  they  also  exhibited  a  number  of 
characteristics  which  were  attributable  to  the  TQ  philosophy  of  the  performance  appraisal 
These  characteristics  include  employee  collaboration  on  performance  goals,  evaluation 
criteria  based  on  teamwork  and  performance  in  teams,  reward  based  on  organization/team 
performance,  evaluation  based  on  process  improvement,  and  focus  on  employee 
development. 

Seven  of  the  organizations  in  this  study  indicated  that  individuals  were  involved  in 
determining  performance  goals  used  in  appraisals.  Only  one  organization  indicated  that 
the  supervisor  determined  performance  goals  100  percent  of  the  time.  The  fact  that 
individuals  participate  in  setting  performance  goals  is  indicative  of  an  appraisal  promoting 
employee  development 


The  evaluation  criteria  for  a  number  of  the  organizations  included  categories  for 
measuring  performance  on  teams  and  team  works.  Additionally,  three  of  the  organizations 
in  the  study  stated  dint  reward/recognition  was  based  on  team  or  department  performance 
as  opposed  to  individual  performance  alone.  Another  evaluation  criteria  which  appears  to 
promote  TQ  is  the  evaluation  of  process  improvement.  Several  respondents  included 
process  improvement  as  one  of  the  areas  assessed  in  their  performance  appraisal 

One  of  the  major  goals  for  performance  appraisal  expressed  by  a  number  of 
respondents  was  the  focus  on  employee  development  in  the  performance  appraisal.  For 
example,  some  of  the  comments  portrayed  the  appraisal  as  being  “strictly  development 
based  and  does  not  rate  performance”,  and  “the  assessment  is  done  on;  provided  a 
platform  for  setting  future  goals.”  This  focus  on  employee  development  is  future-oriented 
(as  opposed  to  measuring  the  past)  and  is  consistent  with  the  goals  of  TQ. 

The  final  category  of  open-ended  responses  includes  information  that  is  characteristic 
of  the  majority  of  the  participant  organizations  in  the  study,  but  is  not  solely  attributable  to 
either  the  traditional  or  TQ  appraisal  philosophies.  These  characteristics  include:  all  of  the 
organizations  use  armual  reriew  periods,  except  for  organization  J,  which  uses  a  6-month 
period  for  non-exempt  employees;  seven  of  the  organizations  use  different  appraisal 
systems  for  managerial  versus  line  employees;  all  of  the  organizations  have  performance 
feedback  systems,  seven  of  the  organizations  provided  less  than  4  hours  training  to  raters 
regarding  the  appraisal  process;  and  all  of  the  organizations  provided  less  than  four  hours 
of  training  to  employees  regarding  the  rating  and  development  process.  Of  these 
characteristics  the  one  that  stands  out  is  the  small  amount  of  training  given  to  both  raters 


60 


and  ratees  regarding  the  appraisal  training  process  In  fact,  when  asked  what  changes 
would  they  like  to  see  with  regard  to  their  appraisals  three  respondents  indicated  they 
would  like  to  see  more  emphasis  on  training  individuals  on  the  day-to-day  use  of  the 
process. 

Investigative  Question  Number  Foun  Which  characteristics  aitd  uses  are  viewed  as 
most  significant  by  the  expert  practitioners?  To  answer  this  question  we  used  two  series 
of  weighted  questions  to  assess  what  the  experts  viewed  as  the  most  important  uses  of 
performance  appraisal  in  the  TQ  environment.  The  first  series  was  asked  in  the  round  two 
questionnaire  and  the  mean  responses  are  depicted  in  Figure  6,  on  the  following  page. 

From  these  responses,  the  uses  receiving  the  lowest  weights  and  a  high  degree  of 
consensus  were  eliminated  fi'om  consideration  for  the  third  round  and  the  remainder  of  the 
alternatives  were  included  in  the  round  three  questionnaire.  The  data  fi-om  the  round  three 
responses  is  represented  in  Figure  7  on  page  63 .  From  the  results  in  Figure  7  it  is  apparent 
that  future  development  was  identified  as  the  most  important  use  for  perfom  ^ice 
appraisal.  In  fact,  all  but  one  of  the  respondents  identified  employee  development  as  what 
they  thought  was  the  most  important  use  for  performance  appraisal  The  second  and  third 
most  important  uses  for  performance  appraisals  are  to  make  compensation  decisions  and 
to  promote  employee  empowerment,  respectively 
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USES  FOR  PERFORMANCE  APPRAISAL 


Used  to  justify 
compensation 


Used  to  foster  employee 
empowerment 


Used  to  foster  teamwork 


Used  for  making 
compensation  decisions 


Used  for  future  development 
of  the  ratee 


Used  for  making  dismissal 
decisions 


Used  for  making  layoff 
decisions 


Used  for  determining 
promotable  employees 


MEAN 


Fiuure  6,  Round  Two  Expert  Opinion  on  Uses  for  Performance  Appraisal 


APPRAISAL  USES 


Used  te  foster 

wnpigyies  xnyowsiTTiiSl 

Used  to  foster 
leamwork 

Used  fcrmaicing 
compensation  dectsions 

Used  as  a  tool  for 
future  devetopmem 

Used  for  determmtng 
promotatile  employ  net 


Figure  7,  Round  Three  Expert  Opinion  on  Uses  for  Performance  Appraisal 


Conclusion 

It  would  appear  that,  with  one  exception,  the  organizations  in  this  study  use  characteristics 
of  the  both  performance  appraisal  philosophies  and  TQ  philosophies  to  design  and 
implement  performance  measurement  in  their  organizations  Additionally  no  two 
organizations  appear  to  integrate  the  two  philosophies  in  the  same  manner  Chapter  V 
addresses  how  the  analyzes  used  to  answer  the  investigative  questions  also  answers  the 
research  question  identified  in  Chapter  I.  Possible  routes  for  future  research  are  also 
presented. 
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V:  Coodusioiu  and  Recommendations 


This  chapter  begins  by  answering  the  fundamental  question  of  this  research  efTon 
which  is,  if  quality  experts  such  as  W.  Edwards  Deming  maintain  that  that  the  performance 
appraisal  is  incompatible  with  TQM,  how  do  organizations  that  have  been  recognized  for 
TQM  excellence  use  and  view  the  performance  appraisal?  The  answei  to  the  research 
question  is  presented  in  terms  of  what  Traditional,  TQ  and  other  characteristics  were 
found  in  the  organizations.  The  chapter  will  then  conclude  by  offering  recommendations 
for  further  research. 

Answer  to  the  Kesenrch  Problem 

Existence  of  Traditional  Appraisal  Practices.  The  scaled  questions  on  traditional 
appraisal  practices  (from  round  one  of  the  Delphi  study)  show  agreement  that  these 
organizations,  except  for  Organization  J,  do  use  traditional  appraisal  techniques.  These 
include  but  are  not  limited  to  using  forced  ratings  scales,  linking  compensation  to 
performance  appraisal,  and  linking  promotion  to  performance  appraisal.  This  is  further 
evidenced  by  the  consensus  on  the  use  of  traditional  appraisal  criteria  such  as  individual 
output  and  more  subjective  criteria  such  as  personal  image  and  ability  to  communicate, 
which  more  than  half  of  these  organizations  include  as  evaluation  criteria.  Furthermore, 
the  appraisals  in  the  majority  of  the  organizations  are  specifically  used  to  hold  individuals 
accountable  and  reward  personnel  based  on  individual  performance,  both  of  which  are  not 
directly  consistent  with  the  TQ  related  literature  regarding  performance  appraisals 
Although  these  organizations  use  traditional  appraisal  practices,  most  organizations  cited 
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either  insufficient  rater  training  or  a  laclc  of  training  altogether  as  a  major  problem  with 
their  appraisil  systems. 

Eiistcncc  of  TQ  Related  Practices.  On  the  other  hand,  these  organizations  do 
incorporate  a  number  of  practices  which  are  consistent  with  the  literature  on  TQ  There 
was  a  significant  range  in  the  responses  of  the  scaled  questions  in  round  two  of  the  Delphi 
study  which  contained  the  TQ  constructs.  This  indicated  that  there  were  differences 
among  organizations  concerning  the  existence  of  performance  appraisal  elements  that  are 
related  to  TQ  philosophies  This  range  was  also  evident  in  the  open-ended  responses; 
however,  the  open-ended  responses  contained  a  number  of  attributes  which  were  relatively 
consistent  among  the  organizations  and  consistent  with  the  TQ  philosophy.  These 
included  joint  determination  of  performance  goals,  teamt/department  performance-based 
con^tensation,  evaluation  based  on  team/group  perfonnance,  and  established  feedback 
policies.  Additionally,  most  organizations  have  adapted  their  appraisal  formats  in 
accordazKX  with  TQ  proponents’  recommendations  These  changes  include  reducing  the 
number  of  rating  categories,  elimination  of  ratings  and  scales  altogether,  and  stressing  the 
use  of  more  objective  criteria  with  the  inclusion  of  mutual  goal  setting  between  supervisor 
and  employee  Most  important,  however,  the  overriding  theme  of  these  organizations  was 
that  the  major  focus  of  the  performance  appraisal  is  to  further  the  development  of  the 
employee.  This  was  also  in  agreement  with  what  the  experts  in  these  organizations 
viewed  as  the  best  use  of  the  performance  appraisal  in  a  TQ  environment  (from  rounds 
two  and  three  of  the  Delphi  study)  Table  14  on  the  following  page  contains  a  summary  of 
the  TQ  related  characteristics 
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TABLE  14,  Varions  Techniques  for  Appraisals  in  a  TQ  Environment 


To  sununarize,  the  organizations  that  participated  in  this  study  incorporate  traditional 
appraisal  practices;  however,  they  ail  incorporate  various  characteristics  in  their  appraisals 
which  are  consistent  with  the  TQM  philosophy,  a  praaice  that  underscores  Bowman’s 
assertion  that  if  organizations  continue  to  use  performance  appraisals,  they  must  adapt 
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them  to  reflect  the  different  environment  of  TQM.  Organization  J  is  the  notable  exception 
as  they  have  eliminated  their  appraisal  in  accordance  with  Demlng’s  guidelines  Another 
exception  may  be  Organization  C  which  uses  appraisals  almost  exclusively  for  the  ftiture 
development  of  employees. 

By  comparing  the  results  of  a  1984  survey  of  how  members  of  the  American 
Management  Association  use  the  appraisal  (located  in  Table  1.  page  9)  with  how  the 
Baldrige  Award  winning  companies  actually  use  the  appraisal  (located  in  Figure  5,  page 
57),  it  is  apparent  that  the  appraisal  still  plays  a  large  role  in  making  compensation 
decisions.  However,  the  use  of  the  appraisal  for  future  development  of  the  employee 
seems  to  be  more  important  today  than  it  was  in  1984.  Using  the  appraisal  to  make 
promotion  decisions  appears  to  have  the  same  importance  today  as  it  did  in  1984. 

By  comparing  the  results  of  the  1984  survey  to  the  organizations*  experts’  opinions  as 
to  the  best  uses  for  the  appraisal  in  a  TQ  environment  (located  in  Figure  6,  page  62),  it  is 
apparent  that  using  the  appraisal  for  future  development  of  the  ratee  takes  on  far  more 
importance  than  using  the  appraisal  for  making  compensation  decisions  and  for  making 
promotion  decisions  Additionally,  the  experts’  opinions  place  more  importance  on  the 
use  of  the  appraisal  to  foster  employee  empowerment  and  teamwork  when  compared  to 
how  their  organizations  currently  use  the  appraisal  for  those  purposes. 

Recommendations  for  future  research 

This  study  has  attempted  to  describe  the  performance  appraisal  practices  of  companies 
that  are  successful  in  the  TQ  environment  It  has  not,  however,  attempted  to  assess  the 
impact  of  these  appraisal  systems  in  terms  of  infusing  TQM  principles  into  the  culture  of 
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the  organization.  This  may  be  an  area  of  interest  for  future  research.  Another  avenue  for 
future  research  might  be  identifying  how  the  employees  in  these  companies  view  the 
appraisal  practices  of  their  organizations.  A  third  avenue  for  future  research  could  be 
identifying  possible  differences  between  the  ways  TQ  companies  and  non-TQ  companies 
view  and  use  their  performance  appraisals.  Finally,  a  fourth  possibility  for  future  research 
in  this  area  may  be  to  identify  what  members  of  organizations  (both  managers  and 
employees)  view  as  the  most  important  characteristics  of  the  performance  appraisal  to 
foster  employee  empowerment  and  TQ.  This  research  has  shown  that  there  are  significant 
dififerences  in  the  way  these  companies  implement  the  performance  appraisal.  Research  to 
identify  which  performance  appraisal  characteristics  are  most  effective  may  also  be 
beneficial. 

Conclusion 

The  results  of  this  study  appear  to  be  mixed  The  participating  organizations  exhibit 
characteristics  of  both  the  traditional  and  TQ  appraisal  philosophies.  The  common  theme 
derived  fi'om  this  study’s  results  is  that  these  organizations  have  attempted  to  infuse 
elements  into  their  performance  appraisal  systems  to  reflect  the  different  working 
environment  that  must  exist  under  the  TQM  pliilosophy.  These  common  elements  include 
using  the  appraisal  for  the  purpose  of  employee  development,  making  use  of  joint  goal¬ 
setting,  eliminating  rating  scales,  incorporating  structured  feedback  programs,  and  making 
objective  criteria  consistent  'with  overall  organizational  goals  and  customer  needs. 

The  r  esults  of  this  study  provide  support  for  the  notion  that  even  though  orgaitizations 
may  be  required  to  administer  performance  appraisals  (due  to  overall  corporate  policy  or 
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by  Congressional  mandate  as  is  the  case  with  the  military  and  other  governmental 
agencies),  the  pertbrmance  appraisal  can  incorporate  elements  that  will  support  the  TQ 
process. 
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APPENTDIX  A:  Dcmtng’s  14  Points  (Deming,  1986:23-96) 

1  Create  constancy  of  purpose  and  management  commitment  Businesses  are  social 
systems  whose  purpose  is  to  serve  its  customers  and  employees.  In  order  to  do  this, 
management  must  make  a  long-term  commitment  to  invest  in  innovation,  training  and 
research.  The  traditional  emphasis  on  short-term  profits  has  taken  its  toll  on  American 
business. 

2.  Adopt  tfte  new  philosophy.  The  Western  system  is  based  on  Scientific  Management 
principles,  which  has  led  to  quotas  for  performance  measurement  and,  as  a  result,  has 
fostered  adversarial  work  relationships.  Adopting  this  new  philosophy  requires  a 
gustnniiy-nnMited  approach  and  cooperation  between  labor  and  management 

3.  Understand  inspection.  Quality  by  inspection  encourages  defectives  because  it  is  up  to 
someone  else  down  the  line  to  catch  mistakes  Workers  must  be  responsible  for  their  own 
work  to  prevent  defectives  from  being  introduced  into  the  system.  Quality  doesn’t  result 
from  inspection.  Rather,  it  results  from  improvement  in  the  process. 

4.  End  the  practice  of  awarding  business  based  on  price  tag  alone  Purchasing  functions 
should  seek  the  best  quality  and  work  to  achieve  it  with  a  single  supplier  for  any  one  item 
in  a  long-term  relationship 

5  Improve  constantly  American  management  has  typically  viewed  improvement  on  a 
large  scale  basis  of  innovation.  The  focus  should  be  on  continuous,  small,  incremental 
improvements  that  reduce  causes  of  variation  and  permit  stability 
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6.  Institvte  training  Employees  must  have  the  proper  tools  and  knowledge  to  p^i-form 
well  and  it  is  the  responsibility  of  management  to  provide  these.  This  also  enhances 
worker  morale  by  showing  that  the  organization  is  committed  to  worker  development. 

7.  institute  leadership.  The  job  of  the  supervisor  is  to  lead.  They  should  be  coaches,  not 
policemen.  Leadership  can  eliminate  fear  and  promote  teamwork. 

8.  Drive  out  fear.  When  employees  are  aftaid  to  ask  questions  or  take  a  petition,  the 
hope  for  process  improvement  is  lost.  Also,  fear  resulting  fron,  organizational  instability, 
such  as  the  threat  of  losing  one’s  job,  causes  short-term  thinking  and  suboptimization. 

9.  Breakdown  barriers  between  staff  areas.  Different  departments  often  compete  against 
each  other.  When  their  goals  conflict  with  the  overall  goals  of  the  organization,  they  do 
not  work  as  a  team  and  cannot  foresee  problems.  When  this  is  the  case,  one  department’s 
goals  may  be  to  the  detriment  of  another. 

10.  Eliminate  slogans  and  targets  for  the  workplace.  These  are  superficial  and  do  not 
motivate  a  worker  toward  an  organization’s  goals 

11.  Eliminate  quotas  and  Management  by  Objectives  (MBOj.  Numerical  quotas  reflect 
short-term  perspectives  and  do  not  encourage  long-term  improvement  The  typical  MBO 
system  focuses  on  results,  not  processes. 

12.  Remove  barriers  to  pride  in  workmanship  People  essentially  want  to  do  a  good  job 
and  are  frustrated  when  the  system  inhibits  them  from  doing  so.  Poor  supervisors,  poor 
training  and  development,  faulty  equipment  and  defective  materials  must  net  be  part  of  the 
system. 
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13  Institute  education  and  training  This  typ-i  of  education  refers  to  self-development 
and  learning, 

l4.  Take  action  to  accomplish  the  transformation  This  requires  much  effort. 
Management  must  be  committed  to  making  this  happen.  It  involves  management’s  freeing 
themselves  of  the  old  paradigm  of  American  management  and  concentrating  on  “moving 
and  refrcezing”  this  cultural  change  throughout  the  entire  organization. 
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APPENDIX  B:  Deming’s  Seven  Deadly  Diseases  (Denting,  1986:97-153) 

1.  Lack  of  comtaticy  of  purpose.  If  a  company  lacks  constancy  of  purpose,  it  has  no 
long-range  plans  for  staying  competitive.  Therefore,  both  managers  and  employees  are 
insecure  and  concentrate  on  short  term  results  in  an  attempt  to  justify  their  positions. 

2.  Emphasis  on  short-term  profits  This  undermines  productivity,  continuous 
improvement  and  quality 

3.  Evaluation  by  performance,  merit  rating  or  annual  review*’  This  nourishes  short-term 
performance,  builds  fear  and  demolishes  teamwork.  Performance  appraisals  focus  on  the 
end  product,  not  the  process  and  not  on  leadership  to  help  the  employees.  Since 
evaluations  are  primarily  made  on  an  individual  basis,  teamwork  is  stifled  Deming  also 
maintains  that  a  fair  rating  is  impossible  because  one’s  performance  is  affected  by  many 
factors,  most  of  which  are  systemic,  and  consequently,  out  of  the  employee’s  own  control. 
4  Mobility  of  management.  Job-hopping  managers  do  not  understand  the  processes 
within  the  organizations  in  which  they  work.  Since  a  manager’s  “life  span”  within  a  job  is 
short,  long-term  planning  is  not  a  priority.  Consequently,  short  term  objectives  that  will 
provide  quick,  dramatic  results  are  the  focus. 

5.  Running  a  company  on  visible  figures  alone  Managers  must  consider  the  intangibles 
that  contribute  to  the  bottom  line  sucn  as  customer  satisfaction,  pride  in  workmanship,  or 
morale.  These  have  far-reaching  affects  and  should  be  considered  by  management. 
Deming’s  deadly  diseases  6  and  7  pertain  expressly  to  American  business. 

6.  Excessive  medical  costs 

7  Excessive  costs  of  warranty  fueled  by  lo'wyers  that  work  on  contingency  fee. 
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APPENDIX  C:  Delphi  First  Round 


This  questionnaire  is  the  first  of  three  rounds  of  questions.  Its  purpose  is  to  provide  us 
with  general  information  pertaining  to  your  organization  and  your  performance  appraisal 
system.  Please  keep  in  mind  that  we  are  interested  in  the  appraisal  practices  used  for 
evaluating  nonmanagerial  employees  and  first-level  supervisors.  This  round  of  questions 
will  be  completed  by  several  expens,  such  as  yourself,  and  the  results  will  be  compiled  and 
redistributed  to  you  in  a  subsequent  survey  round.  Your  input  is  critical  to  our  research. 
Please  complete  this  round  of  questions  as  accurately  as  possible.  Your  responses  will  be 
anonymous  and  will  only  be  used  for  the  purposes  of  this  research  effort.  Please  return 
these  questions  via  fax  no  later  than  seven  calendar  days  after  receipt.  Because  we 
understand  that  faxes  occasionally  get  '‘lost”  during  the  uansmittal  and  routing,  we  will 
get  in  touch  with  you  after  the  seven  day  period  to  make  sure  you  received  it.  Our  fax 
number  is  (513)  476-7988  Thanks  again  for  your  participation 


A.  General  Information; 

Name; _ 

Organization; _ 

1.  Approximately  how  many  employees  are  in  vcur  company*^ 


2.  Are  vour  employees  represented  bv  a  union? 

3  .  How  is  vour  company  disr-jbuted  geographically  ‘’(circle  those  that  apply) 

a.  within  one  community 

b.  more  than  one  cotnmunir>' 

c.  within  one  slate 

d.  more  than  one  state 

4.  What  is  vour  organization's  mission'’  (choose  those  that  apply) 
a.  service 
b  manufacturing 

c.  transportation 

d.  other  (please  specify) _ 

5  What  is  vour  experience  in  the  following  areas  (in  vearsi'’ 

Personnel  or  Human  Resource  Management  in  general. 

T raining  and  Development  _ 

Total  Quality  Management  _ 

Performance  Appraisal  Systems  _ 
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B.  Perfomunce  Appraisal  Related  Issues  as  They  Apply  to  Your  Organization: 

Please  use  the  following  scale  for  questions  1  through  20  of  this  section  (circle  one): 


1 _ 2 _ 3 _ 4 _ 5 

strongly  neither  agree  strongly 

disagree  nor  disagree  agree 

1.  1  2  3  4  5 .  Measurable  objectives  are  established  for  each  employee. 

2.  1  2  3  4  5  .  The  appraisal  process  is  linked  to  employee  development  programs. 

3.  1  2  3  4  5 .  Our  organization  has  established  education  programs  to  effectively 

train  raters  about  performance  appraisal  processes. 

4.  1  2  3  4  5  .  Our  organization  has  establi^ed  education  programs  to  train  rater 

about  performance  appraisal  purposes. 

5.  1  2  3  4  5  .  Our  organization’s  appraisal  process  is  linked  to  reward  decisions. 

6.  1  2  3  4  5  .  Our  organization’s  appraisal  process  is  linked  to  promotion  decisions. 

7.  1  2  3  4  5..  Our  organization’s  appraisal  is  linked  to  layoff  decisions 

8.  1  2  3  4  5  .  Our  organization’s  appraisal  is  linked  to  dismissal  decisions. 

9.  1  2  3  4  5  .  Our  p«formance  appraisal  is  used  as  a  tool  for  improving 

performance 

10.  1  2  3  4  5 .  Our  performance  appraisal  is  used  as  a  tool  for  improving 

productivity. 

11.1  2  3  4  5 .  Our  performance  appraisal  is  used  as  a  tool  for  facilitating  employee 

training. 

12.  1  2  3  4  5  .  Our  performance  jqjpraisal  is  used  as  a  tool  for  facilitating  employee 

development. 

13.  1  2  3  4  5 .  Our  performance  appraisal  is  used  as  a  tool  for  identifying 

promotable  employees. 

14.1  2  3  4  5  .  Our  performance  appraisal  measures  group  or  team  performance. 

15.  1  2  3  4  5 .  Our  performance  appraisal  measures  individual  fjerformance. 

16  1  2  3  4  5 .  Our  appraisal  is  designed  to  evaluate  performance  in  terms  of 

contribution  to  process  improvement 

17.  1  2  3  4  5  .  Our  appraisal  is  designed  to  evaluate  performance  in  terms  of 

contribution  to  the  end-product  (such  as  production  quotas) 
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5 


strongly  neither  agree  strongly 

disagree  nor  disagree  agree 

18.1  2  3  4  5 .  The  performance  criteria  as  evaluated  with  your  appraisal  is  tied  into. 

and  consistent  with,  the  overall  business  goals  of  the  organiaation. 

19.1  2  3  4  5  ....  Our  performance  appraisal  is  conducive  to  fostering  employee 

empowerment. 

20,  1  2  3  4  5  .  Appraisal  ratings  are  specifically  tied  to  performance  based  on 

written  job  descriptions. 


yes  /  no  If  yes,  please  briefly  explain  the  differences. 


24.  Are  the  performance  appraisals  collected  in  a  centralized  location'^  yes /no 

25.  Are  standardized  scoring  scales  used*^  yes /no.  If  yes,  please  bnefly  describe. 

26.  Do  you  have  a  structured  performance  feedback  policy'^  ves  /  no  II  ves.  please 
explain. 


Forced  scales;  _ ^%;  Narrative:  _ % 


29.  Please  specify  who  typically  determines  the  performance  goals  that  are  used  in  your 
organization’s  appraisal(s). 

Supervisor  determines  the  goals _ %  of  the  time;  employee _ %  of  the  time; 

standardized,  corporate  goals _ of  the  time. 

30.  Are  the  following  criteria  evaluated  in  your  performance  appraisal  ?  (circle  yes  or  no) 

a.  employee  output;  yes  /  no 

b.  absences;  yes /no 

c.  overtime:  yes  /  no 

d.  reprimands:  yes  /  no 

e.  philanthropic  activities;  yes  /  no 

f  personal  image;  yes  /  no 

g.  leadership  ability;  yes  /  no 

h.  ability  to  communicate:  yes /no 

i.  others,  please  specify: 
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Please  read  the  following  passage  in  order  to  answer  questions  33a  and  33b. 

One  of  W  Edwards  Deming’s  Seven  Deadly  Sins  that  he  believed  plagues  Western 
organizations  is  evaluation  by  performance,  merit  rating,  or  annual  review.  “These  types 
of  evaluations  nourish  short-term  performance,  build  fear  and  demolish  teamwork 
Performance  appraisals  focus  on  the  end  product,  net  the  process  and  not  on  the 
leadership  to  help  the  employees.  Since  e'/aluations  are  primarily  made  on  an  mdividuai 
basis,  teamwork  is  sdfied." 


_ 1 _ 2 _ 2 _ 4 _ 2 _ 

strongly  neither  agree  strongly 

disagree  nor  disagree  agree 

33a.  1  2  3  4  5  .  My  organization’s  appraisal  philosophy  is  representative  of  this 

“deadly  sin”. 


33b,  Please  describe  how  vour  organization’ .s  appraisal  system  is  or  is  not  representative 
of  this  “deadly  sin”. 
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Appaidix  D:  Round  Two  Questionnaire 


Ddphi  Study  Round  Two 


The  purpose  of  this  round  of  questions  is  to  provide  you  with  feedback  on  round  one 
responses,  and  to  further  define  t^  performance  appraisal  under  TQ.  Section  1  contains 
the  feedback  on  round  1  responses.  Section  2  contains  scaled  questions  and  a  rank  order 
question  investigating  preferred  TQ  oerformance  techniques.  It  is  designed  to  further 
define  TQ  appraisal  approaches.  This  questionnaire  should  take  no  more  than  15  minutes 
of  your  time.  Please  return  the  last  three  pages. 


SECTION  i:  Round  One  Feedback. 

Below  is  a  table  containing  the  results  of  the  scaled  responses  obtained  in  round  one.  The 
second  table  is  a  depiction  of  the  responses  to  the  yes/no  responses.  Also  included  is  a 
histogram  of  the  responses  to  the  scaled  questions. 


TABLE  1:  Traditional  Appraisal  Constructs 


I 

Measurable  objectives  are  established  for  each  etnplov'ee 

4.22/1.09 

The  appraisal  process  is  linked  to  employee  de\'elopment  programs. 

4.33/1.32 

Our  organizalion  has  education  programs  to  efiecovely  tram  raters  about 

perfomiance  appraisal  processes. 

3.22/1.39 

Our  organizauon  has  established  education  programs  to  tram  rater  about  performance 
appraisal  purposes. 

3.55/1.23 

Our  organizations  appraisal  processes  is  linked  to  reward  decisions. 

4.11/1.45 

Our  orf'ani’ai'ons  appraisal  process  is  linked  to  promotion  decisions. 

4.11/1.16 

Our  organizations  appraisal  is  linked  to  layoff  decisions 

3  66/T.66 

Our  organizations  appraisal  is  linked  to  dismissal  decisions 

4.55/1.33 

Our  perfomiance  appraisal  is  used  as  a  tool  for  improving  performance 

4.77/44 

Our  Derformance  appraisal  is  used  as  a  tool  for  improving  productivitv 

4.55/1.01 

4.22/1.09 


Our  perfonnancc  appraisal  is  used  as  a  tool  for  faciiitatinj;  emplov’ce  training 

Our  performaiKC  appraisal  i$  used  as  a  tool  for  facilitating;  employee  dev-elopment 

4.44/1.01 

2.66/1.58 

Our  perfomiance  appraisal  measures  indhiduai  performance 

4.77/44 

Our  appraisal  is  designed  to  esaluate  performance  in  terms  of  contribution  to  the  end 
product  (such  as  production  quotas). 

3.77/1.39 

The  perfoimaiicc  criteria  as  es’aluated  vrith  your  appraisal  is  tied  into,  and  conststent 
«ith.  the  overall  business  goals  of  the  organization. 

4.66/.71 

Appraisal  ratings  are  specihcalh'  tied  to  performance  based  on  tvnnen  job 
desoipdons. 

3-11/1.45 

TOTAL: 

4.26/1  32 

TABLE  2:  answers  to  ycs/no  qiiestions 


Qaestioa: 

Respoase:  Yei/No 

7/1  (1  uses  form  for  nonunion  oolv) 

Are  perfonnaace  appraisals  collected  in  a 

7/2 

centralized  location? 

Are  standardized  scoring  scales  used? 

6/3 

Do  you  base  a  structured  performance  feedback 

8/1 

policy? 

Are  the  following  cntena  evaluated  in  your 

perfbnnauce  appraisal? 

*cnipioyce  output 

8/1 

-absences 

5/4 

•overtime 

3/6 

reprimands 

3/6 

•philanthropic  acti  vines 

3/6 

•personal  image 

5/4 

-leadership  ability 

9 

-ability  to  communicate 

9 
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Histogram  of  Round  One  Data 


Hiiitoiiraca 


Strongly  Neitber  agree  Strongly 

disagree  nor  disagree  agree 


The  constructs  in  the  table  1  measure  the  existence  of  traditional  performance  appraisal 
practices  in  the  various  organization  participating  in  the  study  Table  2  also  measures 
traditional  appraisal  policies/practices.  The  histogram  is  a  summary  of  the  responses 
across  all  the  scaled  questions.  Based  on  these  responses  it  is  evident  that  most  of  the 
companies  use  many  traditional  approaches  to  performance  appraisal,  however,  it  is  also 
evident  from  the  responses  to  the  open  ended  questions  that  most  organizations  in  the 
study  employ  other  techniques  to  foster  the  TQ  environment  and  promote  employee 
empowerment.  A  summary  of  the  various  approaches  is  outlined  in  table  3  on  the 
following  page. 
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TABLE  3 ;  Various  Techniques  for  Appraisal  Under  TQ 


-Use  of  unique  performance  criteria  including: 

-dedication  to  customers  (intemal/extemal) 
-quality  of  output 

-initiative  (i.e.  process  improvement) 
-versatility  (team  participation) 

_ -reliability _ 

-Unique  goals  of  performance  appraisal  including: 

-to  recognize  contribution  through  performance  planning  intensive  feedback  and 
review. 

-to  empower  employee  performance  and  development.  Empower  to  generate  action  in 
service  of  their  commitments 

-to  align  individual  performance  with  department/function/division/'company  goals 
-to  hold  ail  individuals  responsible  and  accountable  for  performance  and  development 
-to  promote  employee  development  and  not  track  a  report  card  -not  used  to  drive 

salary  promotion  or  selection _ 

-Tttdiniques  to  promote  team  performance  and  process  improvement  include 
-bonus  system  based  on  perfoimance  as  a  team  as  opposed  to  individual 
-team  reviews  and  compensation 
-recognition  based  on  team  accomplishments 
•team  member  input  is  sou^t  on  appraisals 
-process  management  as  well  as  results  are  included  in  appraisals 
-individual  and  team  define  the  specific  performance  parameters/ goals 
-appraisal/assessment  is  done  only  to  provide  a  platform  for  setting  future  goals 
-strictly  development  based  appraisals  which  do  not  rate  performance 

-forward  looking  vs.  backward _ 

-Unique  empowerment  techniques  include. 

-Set  expectation  at  beginning  of  year  and  turn  employee  loose  to  accomplish  goals 
-measured  as  mitiative  on  appraisal  form  —comments  on  employee  discovering  and 

solving  problems/improving  processes _ 

-Other  techniques  include: 

-more  narrative  i.e.  fewer  forced  scales 
-established  measurement  criteria 
-manaeement  traininc  on  evaluating  for  TO 
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B.  This  section  attempts  to  evaluate  your  organization’s  uses  for  your  performance 
appraisal.  In  column  A,  please  assign  a  weight  to  each  based  on  how  vour  organization 
actually  uses  the  performance  appraisal.  You  have  100  points  which  to  allocate  in  each 
column.  If  your  organization  does  not  use  the  appraisal  for  one  of  these  purposes,  you 
should  assign  it  a  weight  of  zero.  Otherwise  please  assign  a  weighting  factor  to  each 
question  based  on  the  degree  to  which  the  appraisal  is  used  for  each.  You  may  assign  any 
amount  of  points  to  each  potential  use  as  long  as  you  only  allocate  100  total  points  for 
each  column.  In  column  B,  please  assign  a  score  in  this  same  manner  based  on  your 
expert  opinion  of  how  appraisals  in  the  Total  Quality  environment  should  be  used. 

A  B 


1 .  _  _  Used  for  determining  promotable  employees 

2.  _  _  Used  for  making  layoff  decisions 

3.  _  _  Used  for  making  dismissal  decisions 

4.  _  _  Used  for  future  development  of  the  raiee 

5.  _  _  Used  to  categorize  the  ratet  jased  on  past  performance 

6.  _  _  Used  for  making  compensation  decisions 

7.  _  _  Used  to  foster  teamwork 

8.  _  _  Used  to  foster  employee  empowennent 

9  _  _  Used  to  evaluate  performance  based  on  objective  criteria 

10.  _  _  Used  to  evaluate  performance  based  on  subjective  criteria 

1 1 .  _  _  Used  to  justify  compensation 

100  Too  Total 

Please  list  anv  other  uses  that  vou  feel  should  be  incorporated  into  the  third  round  of 
questions.  Provide  a  numerical  weight  for  it  based  on  the  instructions  above,  but  DO 
NOT  include  this  weight  in  the  scales  above 
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and  the  relative  imponance  of  each 


Please  state  how  much  formal  training  raters  receive  regarding  t.h<>  appraisal  and  the 
rating  process 


3.  Please  state  how  much  formal  training  the  ratees  receive  regarding  the  appraisal  and 
the  rating  process 


4.  *Axlditional  Conunents/Suggestions: 


if 
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Appendix  E:  Delphi  Quesdonnaiie  Round  Thrij« 


A. 

The  puqjose  for  this  section  is  to  attempt  to  gain  consensus  on  these  questions  that 
appeared  in  round  two.  Only  the  responses  that  liave  not  gained  consensus  of  the 
oarticipants’  opinions  are  included.  You  have  50  total  points  with  which  to  allocate.  The 
higher  the  number  allocated,  the  more  important  its  use  is  in  a  TQ  environment.  Rank 
order  these  uses  for  the  appraisal  based  on  your  expert  opinion  of  the  best  uses  of  the 
appraisal  within  a  TQ  environment.  In  this  section,  we  want  to  know  your  opinion,  not 
the  w.ay  it’s  used  in  your  company, 

_ Used  for  determining  promotable  employees 

_ Used  as  a  tool  for  fiiture  development  of  the  ratee 

_  Used  for  making  compensation  decisions 

_  Used  to  foster  teamwork 

_ Used  lo  fusici  cuiployee  empowerment 


SO  Total 

B. 

Please  respond  to  the  folio wng  questions  based  on  the  scale  below. 

1  2  3  4  5 


strongly  neither  agree  strongly 

disagree  nor  disagree  agree 

1.  1  2  3  4  5 . Our  organization  uses  peer  ratings  as  opposed  to  supervisor  ratings 

2.  1  2  3  4  5 . Our  organization  uses  peer  ratings  in  addition  to  supervisor  ratings 

3.  1  2  3  4  5 . Performance  goals  are  mutually  agreed  upon  between  rater  and  ratee 


Appendix  F:  SAS  program  for  computing  Cronbach’s  Alpha 


Program  for  Traditional  Appraisal  Questions 

options  linesize=80; 
data  tests ; 

infile  thesis  1  missover; 

input  ql  1  q2  2  q3  3  q4  4  q5  5  q6  6  q7  7  q8  8  q9  9  qlO  10  qll  11  ql2  12 
ql3  13ql4  14ql5  15  ql6  16ql7  17ql8  18  ql9  19  q20  20  q33a  21; 

proc  means  mean  stdev; 

var  ql  q2  q3  q4  qS  q6  q7  q8  q9  qlO  qll  ql2  ql3  ql4  ql5  ql6  ql7  ql8  ql9 
q20  q33a; 

Proc  corr  alpha; 

var  ql  q2  q3  q4  q5  q6  q7  q8  q9  qlO  qll  ql2  ql3  ql5  ql7  ql8  q20; 


Program  for  TQ  Appraisal  Questions 

options  linesize=80; 
data  rtestS; 

infile  round2  missover; 

input  itl  I  it2  2  it3  3  it4  4  it5  5  it6  6  it7  7  it8  8  it9  9  itlO  10  itl  1  11 
itl2  12  itl3  13  itl4  14  itl5  15  itl6  16  itl7  17  itl8  18  itl9  19  it20  20 
it21  21  it22  22  it23  23  it24  24  it25  25-26  it26  27-28  it27  29-30  it28 
31-32  it29  33-34  it30  35-36  it31  37-38  it32  39-40  it33  41-42  it34  43-44 
it35  45-46  it36  47-48  it37  49-50  it38  51-52  it39  53-54  it40  55-56  it41 
57-58  it42  59-60  it43  61-62  it44  63-64  it45  65-66  it46  67-68; 

Proc  corr  alpha; 

var  itl  it2  it3  it4  it5  it6  it7  it8  it9  itlO  itl  1  itl2  itl3  itl4  itl 5 
itl6  it  17  it  18  it  19  it20  it21  it22  it23  it24; 


CroQbach’s  alpha  for  Traditional  Constructs 


Correlation  Analysis 
Cronbach  CoefBcient  Alpha 


for  RAW  variables  :  0,840522 
for  STANDARDIZED  variables:  0  845594 


Raw  Variables  Std.  Variables 


Deleted 

Variable 

Correlation 
with  Total 

Alpha 

Correlation 
with  Total 

Alpha 

Qi 

0,611847 

0.824345 

0.630255 

0.828328 

Q2 

0,407493 

0.834386 

0.433300 

0  S3&440 

Q3 

0.158916 

0.849001 

0.171271 

0.851245 

Q4 

0.489590 

0  829785 

0.498228 

0.835153 

Q5 

0.557214 

0.825553 

0.567026 

0.831621 

Q6 

0.600914 

0.824242 

0.603860 

0.829708 

Q7 

0.725204 

0.813223 

0.711404 

0.824037 

Q8 

0.444765 

0.832296 

0,403919 

0,839912 

Q9 

0,528681 

0.835556 

0,525708 

0,833748 

QIO 

0.288758 

0  839220 

0,336003 

0.843280 

Qll 

0,379988 

0.835332 

0.430851 

0.83S563 

Q12 

0.580096 

0.826584 

0,597838 

0,830022 

Q13 

0.759592 

0.811069 

0777618 

0.820480 

Q15 

0,287971 

0,839991 

0,108563 

0.849945 

Q17 

0.344634 

0.838389 

0,313082 

0.844405 

Q18 

0.3 18893 

n  rt  s  t  a 

U.OJOl 

A 

0,846384 

Q20 

0.290823 

0.842105 

0.266148 

0  846692 

Cronbacb’s  alpha  for  TQ  appraisal  constructs 


Correlation  Analysis 


Cronbach  CoefiRcient  .Alpha 

for  RAW  variables  :  0.869565 
for  STAI^ARDIZED  variables.  0.881 692 


Raw  Variables 

Std.  Variables 

Deleted 

Correlation 

Correlation 

Variable 

wUi  Total 

Alpha 

with  Total 

Alpha 

ITl 

0.670937 

0.858624 

0  668317 

0  871322 

IT2 

0.281690 

0.870977 

0.375143 

0.879356 

IT3 

0.408988 

0.865457 

0.370519 

0.879480 

m 

0.176449 

0.871156 

0.173977 

0.884657 

ITS 

0.273507 

0.871875 

0.217169 

0.883533 

IT6 

0.304771 

0,867976 

0.368366 

0.879537 

rvn 

1 1  r 

fV 

0.801656 

A  A  *> 

u.ou  /  J 

ITS 

0.391187 

0.865966 

0.435328 

0.877737 

IT9 

0.583046 

0.859383 

0.580702 

0.873762 

ITIO 

0.891484 

0.851423 

0.905887 

0  864534 

ITU 

-0.0249.56 

0.877316 

-0.009098 

0.889335 

m2 

0.285422 

0  868864 

0.330148 

0.880556 

ni3 

0.679857 

0.857464 

0.674265 

0.871155 

IT14 

0  564138 

0.861436 

0.576244 

0  873885 

IT15 

0.854525 

0.851605 

0.830967 

0.866702 

IT16 

0.650373 

0.859228 

0.654470 

0  871709 

m? 

0.628764 

0.860764 

0.600698 

0.873208 

ITl  8 

-0.174153 

0.884954 

-0  190778 

0  893843 

ITl  9 

0.493197 

0.863677 

0.452945 

0.877260 

IT20 

0.267522 

0.868814 

0.279447 

0.881899 

1121 

0. 7543 10 

0.856482 

0.802928 

0.867507 

IT22 

0.191848 

0.872815 

0.120662 

0.886033 

IT23 

0.335907 

0.867650 

0.310549 

0.881077 

IT24 

0.742279 

0.855592 

0.762087 

0.868673 
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Appendix  G:  Presentation  of  Round  One  Data 

Delphi,  Round  One.  The  round  one  questionnaire  gatnered  information  for  several 
purposes  The  first  section  gathered  general  information  about  the  organization  and  the 
participants.  This  information  was  summarized  in  Chapter  III,  The  second  section 
consisted  of  20  questions  that  generated  responses  from  a  Likert  scale  ranging  from 
Usirongly  disagree)  to  5(strongly  agree).  These  questions  attempted  to  ascertain  the 
extent  to  which  the  participating  organizations  used  the  traditiona.l  approaches  to  their 
appraisals.  The  final  section  comprised  of  open-ended  questions.  The  purpose  for  these 
questions  was  to  gain  more  descriptive  information  about  the  participants’  appraisals. 


The  responses  for  questions  1  through  20  were  based  on  the  following  Likert  scale: 


1 

2 

3 

4 

5 

strongly 

neither  agree 

strongly 

disagree 

nor  disagree 

agree 

Organization 


Question  ABC  DE  FGHI 

1,  Measurable  Objectives  are  established,  for  each 
employee 

51  5 

Ji 

41 

31  5 

1  “ 

5 

5 

2.  The  appraisal  process  is  linked  to  employee 
development  pro.grams 

5 

M 

5 

5 

2 

5 

5 

12 

L_ 

~5 

3.  Our  organization  has  established  education  piograms 
to  effectively  train  raters  about  the  appraisal  processes 

H 

I 

1 

i 

”3 

i 

3 

4.  Our  organization  has  established  education  programs 
to  effectively  train  raters  about  the  appraisal  purposes 

I 

1 

H 

i 

3 

3 

1 

n 

DEI 

m 

E 

EE 

6.  Our  appraisal  process  is  linked  to  promotion  decisions 

[I 

4 

2{  4i  5!  3 

5!  5l  5 

7.  Ou;  appraisal  process  is  linked  to  layoff  decisions 

U  l:  5i  5 

4J  41  5 

8.  Our  appraisal  process  is  linked  to  dismissal  decisions 

51  5' 

1 

5i  5!  5 

1  5 

5i  51 

9.  Our  appraisal  process  is  used  as  a  tool  for  improving 
pefformance 

5 

4 

5! 

] 

4 

5 

'  1 

10  Our  appraisal  is  used  as  a  tool  for  improving 

1 

e 

E 

1  2!  51  5!  5 
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roductivi 


1 1 .  Our  appraisal  is  used  as  a  tool  for  facilitating 
employee  training 


12.  Our  appraisal  is  used  as  a  tool  for  facilitating 
employee  development 


13.  Our  appraisal  is  used  as  a  tool  for  identifying 
promotable  employees 


14.  Our  appraisal  measures  group  or  team  performance 


15.  Our  appraisal  measures  individual  performance 


16.  Our  appraisal  is  designed  to  evaluate  performance  in 
terms  of  contribution  to  process  improvement 


5  4  5  3  5  2  4  5  5 


5  3  2  2  5  3  3 


18.  The  performance  criteria  in  the  appraisal  is  tied  into 
and  consistent  with  overall  goals  of  the  organization 


19.  Our  appraisal  is  conducive  to  fostering  employee 
empowerment 


20.  Appraisal  ratings  are  specifically  tied  to  performance 
based  on  written  job  descriptions 


5  5  4  5  5  5  3  5  5 


1  4  1  3  5  5  3  3  3 


yes;  All  organizations  except  the  two  below 
no:  Organizations  I  and  J 

recommended:  Organization  A  (union  employees  aren’t  required  appraisals) 
22.  What  is  the  tvoical  reviewing  period  in  vour  organization  fi.e..  annuallvl? 


.\11  organizations  who  use  appraisals  have  annual  review  periods  except  for 
Organization  G  which  uses  a  six-month  period  for  non-exempt  employees 

23.  Does  vour  organization  have  different  appraisal  svstems  for  managerial  versus 


line  employees?  yes  /  no  If  yes,  please  briefly  explain  the  differences, 
yes:  Organization: 

A:  union-represented  employees  don’t  have  appraisals 
scales  are  different 

E:  management  has  management  responsibilities  included 

F:  different  values  are  measured  and  managerial  reviews  focus  on  the 
future 

D:  the  rating  scale  is  different  between  the  two;  also,  different 
performance  measures  are  used  for  managerial  employees 

G:  exempt  employees  have  added  managerial  responsibilities 
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H:  managei’  adds  promotability  and  more  detailed  career  planning  and 
development 

24.  Are  the  Performance  appraisals  collected  in  a  centralized  location?  yes  /  no 
All  organizations  responded  yes  except  for  Organizations  A  and  I 

25.  Are  standardized  scoring  scales  used?  yes  /  no.  If  yes,  please  briefly  describe. 
Yes:  Organization 

E:  uses  a  4  point  scale  with  1  being  the  highest 

D;  7-point  scale  for  managers  and  4-pomt  scale  for  non-managers 

H;  4-poini  scale 

All  others:  No 


26.  Do  vou  have  a  structured  performance  feedback  policy?  ves  /  no.  If  yes,  please 
explain. 

Organization 
No:  C 

Yes:  H:  supervisor  and  employee  work  through  the  formal  appraisal  and  must 

reach  consensus 
E:  quanerly,  documented 
B:  quarterly,  documented 
A  supervisors  are  trained  to  be  coaches 
D:  computer  tracked,  documented 
G:  not  mandatory  for  all  line  workers 
F  must  receive 
I.  done  annually  with  appraisal 


27.  Fiease  state  the  rvoical  rype  of  performantc  goal  used  in  your  organization 


Organization: 

B '  no  response 
C:  sales,  goals 

D:  practical,  valid,  and  fair  (productivity  standards) 

■A.  development  due  dates,  quality  and  completion  metrics,  revenue,  cost 
reduction,  budget  performance 
E  narrative,  specific 

I  complete  state  goal  per  cycle,  six  sigma,  and  quality  standards 
G  attainment  of  production  goals 
F  multifaceted  based  on  numerous  factors 


H.  completed  objectives  by  preestabiished  due  dates,  percentage  improvements 


28.  Please  break  down  vour  appraisal  format  in  terms  of  forced  scales  versus 
Fo’ ced  scales:  %  Scales/  Narrative  %; 

Organizabon 
A:  0/1 00 
B:  50/50 
C;  0/100 
D:  30/70 
E:  10/90 
F: 

G.  50/50  (exempt),  90/10  (non-exempt) 

H:  0/100 
I:  0/100 
J:  N/A 


29.  Please  specify  who  typically  determines  the  performance  goals  that  are  used  in 
vour  orsanization’s  appraisaKs). 

Supervisor  determines  the  goals  %  of  the  time/  employee  %  of  the  time/' 
standardized,  corporate  goals  Ve  of  the  time. 

Organization 

A:  50/50/100 
B:  50/50/0 
C:  0/0/0 
D  100/0/0 
E  10/80/10 
F  50/50/100 
G:  100/0/0 

u-  sQ/sn 
I:  50/50/100 
J:  N/A 


30,  Are  the  following  criteria  evaluated  in  your  performance  appraisal  ?  (circle  yes 
or  no) 


a. 

employee  output 

yes 

X  X  X  X  X  XXX 

/  no 

X 

b. 

absences: 

yes 

X  X  X  X  X 

/  no 

X  X  X  X 

c. 

overtime: 

yes 

XXX 

/  no 

X  X  X  X  X  X 

d. 

reprimands 

yes 

X  X  X 

/  no 

X  X  X  X  X  X 

e 

pliilanthropic  activities 

yes 

XXX 

/  no 

X  X  X  X  X  X 
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personal  image; 

yes 

X  X  X  X  X 

/  no 

leadership  ability 

yes 

XXXXXXXXX 

/  no 

ability  to  communicate; 

yes 

XXX. S  XX  X  XX 

/  no 

others,  please  specify; 

~  technical  skills 

X 

—  dedication  to  customers 

X 

—  integrity 

K 

"  teamwork 

X  X 

— qualits'  of  output 

X 

—initiative 

X 

—job  knov/ledge 

X 

—goal  achievement 

X 

—  versatility 

X 

—  reliabilitv 

X 

31.  If  YOU  could  express  the  philosophies  goals  and  practices  of  vour  oerfonnance 
appraisal  in  terms  of  a  “mission  statetnent”.  what  would  it  be? 

Organization: 

F  .  no  response 

G;  to  recognize  contribution  through  perfomnance  planning  intensive 
feedback  and  review. 

H;  provide  feedback  for  employee  to  use  in  continuously  improving 
performance 

I  none 

D  assesses  and  reward  performance  of  the  individual  employee 

A;  to  empower  employee  performance  and  development,  empower  to 
generate  employee  action  in  the  service  of  their  commitments 

B :  no  response 

E.  to  align  individual  and  team  perfoimance  v-ith  department,  function, 
division,  and  company  goals,  to  hold  ail  individuals  responsible  and 
accountable  for  performance  and  development,  to  focus  on  personal 
contribution  to  meeting  the  organization’s  objeaives 

C  based  on  employee  development  and  does  not  track  a  report  card  or 
direction,  ratings  are  not  used  and  the  appraisal  does  not  drive  salary 
promotion  or  selection  (at  least  not  directly),  this  process  is  based  largely 
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f.  personal  image: 

yes 

K  X  X  X  X 

/  no 

g.  leadership  ability 

yes 

xxxxxxxxx 

/  no 

h.  ability  to  communicate. 

yes 

XXX. XXX  X  XX 

/  no 

i.  others,  please  specify: 

~  technical  skills 

X 

~  dedication  to  customers 

X 

~  integrity 

X 

~  teamwork 

X  X 

—quality  of  output 

X 

—initiative 

X 

—job  knov/ledge 

X 

—goal  achievement 

X 

~  versatility 

X 

—  reliability 

X 

31.  If  vou  could  express  the  philosophies  goals  and  practices  of  your  performance 
annraisai  in  terms  of  a  ‘^mission  statement”,  what  would  it  be? 

Oraanization: 

F ;  no  response 

G:  to  recognize  contnbution  through  performance  planning  intensive 
feedback  and  review. 

H  provide  feedback  for  employee  to  use  in  continuously  improving 
performance 

I;  none 

D  assesses  and  reward  performance  of  the  individual  employee 

A.  to  empower  employee  performance  and  development,  empower  to 
generate  employee  action  in  the  service  of  their  commitments 

B  no  response 

E,  to  align  individual  and  team  perfoiraance  with  department,  function, 
division,  and  company  goals,  to  hold  all  individuals  responsible  and 
accountable  for  performance  and  development,  to  focus  on  personal 
contribution  to  meeting  the  organization's  objectives 

C  based  on  employee  development  and  does  not  track  a  repon  card  or 
direction,  ratings  are  not  used  and  the  appraisal  does  not  drive  salary- 
promotion  or  selection  (at  least  not  directly),  this  process  is  based  largely 


96 


I:  We  are  a  pay  for  performance  company  —  always  has  been  and  forecasted  to  be  so. 
Deming’s  pltilosophy  was  considered  in  1983  when  we  designed  our  TQM 
approach  and  was  rejected.  We  still  are  a  pay  for  performance  company. 

G:  We  measure  performance  and  contribution  and  base  our  reward  system  on  that. 
However,  we  also  l.ave  a  bonus  system  that  is  heavily  based  on  the  divisions 
performance  as  a  team.  Nearly  all  our  employees,  exempt  and  non-exempt  are 
bonused.  Additionally  we’re  now  heavily  into  self  directed  work  teams  and  are 
using  a  consulting  firm  to  work  towards  team  reviews  and  compensation.  I  do  not 
expect  we  will  totally  move  away  &om  individual  assessment  and  compensation. 

H;  Team  performance  and  panicipation  is  part  of  both  goals  and  evaluation. 
Recognition  is  almost  entirely  devoted  to  team  accomplishments. 

F;  No  response 

D:  There  is  merit  in  Deming’s  statem.ent  however,  our  corporate  culture  is  such  that 
it  places  great  emphasis  on  individual  initiative  and  performance  we  have  moved 
towards  objective  standards  of  performance  where  possible.  These  are  not  as 
debatable  and  not  a  managers  judgment  that  is  more  liable  to  bias.  We  may  move 
towards  teams  in  the  future,  but  that  does  not  fit  us  that  well  yet. 

E;  Ourappraisal  system  assesses  individual  performance  and  contribution. 

Specific  conformance  objective  are  established  and  are  formally  assessed 
annually.  Merit  pay  is  based  upon  the  annual  performance  numerical  rating  for 
each  individual;  however,  individuals  participation,  interaction,  and  contribution 
on  teams  is  assessed.  Team  member  input  is  sought  on  appraisal.  Process 
management  along  with  end  results  are  included  in  appraisals.  Individuals  and 
teams  define  the  specific  performance  parameters  and  goals.  Regular  open  honest 
communication  is  the  key  to  stifling  fear.  The  Quarterly  progress  reviews 
accomplish  this,  through  face-to  face  interaction  between  the  employee  and  their 
manager. 

B;  Performance  rating/annual  review  only  a  small  part  of  an  overall  process  called 
the  Personal  Performance  Platform.  The  platform  emphasizes  link  between 
individual  and  UCS  goals,  teamwork,  empowerment,  development 

A:  We  don’t  use  ratings  during  the  coaching  conversation.  The  conversation  begins 
with  an  assessment  of  actual  vs  planned  results.  The  assessment  is  done  only  to 
provide  a  platform  for  setting  future  goals  for  both  the  employee  and  supervisor. 

C;  It  is  strictly  development  based  and  does  not  rate  performance.  We  are 
interested  in  the  continues  improvement  of  our  work  force.  We  tend  to  look 
forw'arded  instead  of  backward  when  we  hold  personnel  development  interviews. 


•J:  N/A 


34.  In  what  wavs  does  vour  performance  appraisal  foster  employee  empowenrient'^ 

Organization; 

A;  Teams  are  empowered  to  make  changes 

G;  The  initiative  segment  of  the  appraisal  often  invokes  comments  on  the 
employee  discovering  and  solving  problems  or  gaining  new  knowledge  and 
skill. 

I;  Sets  e\p‘=^ctation  for  outputs  at  beginning  of  year  and  turns  loose  the 
employe-,  to  deliver. 

F:  The  appraisal  document  does  not  factor  into  this.  It  is  determined  on  the 
ability  of  the  manager  to  foster  empowerment,  teamwork  process 
improvement.  ,\lso,  on  the  wants  of  the  manager  have  these  qualities  in 
their  departments. 

B :  Management  and  associates  jointly  set  objectives.  The  manager  keeps  an 
ongoing  record  which  employees  can  see  at  any  time. 

A.  Builds  a  better  working  relationship  between  supervisor  and  employee  as  the 
emphasis  is  on  coaching.  It  is  a  faa-based  assessment. 

E:  Employees  define  specific  measurements  and  goals. 

C;  Encourage  employees  to  take  on  greater  responsibilities 

D;  Reflects  the  job. 


3S.  In  what  wavs  does  vour  performance  appraisal  foster  teamwork? 

Organization: 

H:  Goals,  measurements,  recogitition  and  rewards  depend  on  team  paiticipation. 

G.  The  versatility  segment  speaks  specifically  to  the  employees  flexibility  in 
working  with  others. 

F  No  response 

I:  Some  objectives  may  be  team  created. 
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B:  In  areas  where  individuals  work  as  a  team,  objectives  reflect  accountability 
for  team  performance 

E.  Team  member  input  is  assessed  in  the  appraisal. 

C:  Ratings  are  not  used  so  there  is  no  competition  for  individual  performance 

A;  Teamwork  is  one  of  the  values  that  is  assessed.  Performance  on  a  team  is 
often  cited  as  an  accomplishment  of  a  plan. 

D:  Probably  does  not. 


36.  In  what  wavs  does  your  performance  appraisal  facilitate  process  improvement? 

Organization: 

H;  Teams  work  on  improving  their  process 

G:  Only  indirectly,  we  have  a  large  number  of  quality  improvement  teams  at 
Organization  G  a  part  of  the  QPI  Process  is  to  list  the  steps  of  the  process 
being  revised/altered  then  data  to  show  cause  and  effect  analysis  and  changes 
being  made  in  the  process.  These  are  presented  to  large  management  forums, 
and  participants  are  recogrtized  for  there  contribution  in  their  appraisals 

F:  No  response 

I:  Some  objectives  may  specify  reduction  in  cycle  times,  reduced  errors,  etc., 
which  would  drive  process  improvement  activities. 

A:  The  personal  Performance  Platform  was  developed  by  a  PIT.  In  ntany  cases 
employees’  objectives  are  based  on  process  improvement  participation 

E;  Process  management  is  part  of  the  appraisal.  Every  individual  is  part  of  a 
process 

C:  Look  at  process  improvement  on  as  it  pertains  to  continuous  improvement 

B;  Participation  on  a  PIT  is  cited  as  an  accomplishment 

D;  Feedback  is  given  in  an  attempt  to  focus  on  the  future. 
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37.  How  has  the  current  performance  appraisal  design  changed  in  comparison  to 
the  appraisal  before  vour  company  made  its  transition  into  the  Total  Quality 
Management  environment? 

Organization: 

F:  None 

G:  No  response 

H:  Much  more  emphasis  on  teams  and  on  customer  (internal  and  external) 

I:  Now  tied  into  corporate  priorities/objectives.  Measurement  tools  and  targets 
established  at  beginning  —more  objective,  elimination  of  box  scores.  More 
narrative  w/employee  input  and  review 

B:  Reduced  rating  categories  from  5  to  3.  More  focused  on  feedback  for 
employee  development.  More  focus  on  group  goals 

E;  Teamwork,  process  management  and  customer  focus  have  been  added  to  the 
appraisal.  Employees  have  more  input 

C:  Did  away  with  ratings  in  order  to  be  consistent  with  Dr  Deming’s  philosophy 

A:  Did  away  with  ratings  and  forced  scales.  Conversation  is  in  the  context  of 
comparing  the  actual  results  to  the  planned  for  the  purpose  of  improving  in  the 
future 

D:  Have  tried  to  be  more  objective  by  using  performance  standards 


38.  In  Your  opinioii,  in  what  wavs  are  vour  current  performance  appraisal  policies 
and  practices  consistent  with  the  princiotes  of  Total  Quality  Management? 

Organization: 

F:  Established  measurement  cri:eria  (i.e.  goals)  feedback,  and  problem  solving 
and  recommendations  for  improvement 

G:  Key  elements  of  the  appraisal  form  address  faaors  which  are  essential  to 

quality-factors  like  quality  of  work  (accuracy,  reliability,  and  thoroughness  of 
completed  assignments)  job  knowledge,  productivity  (quickly,  efficiently,  and 
on  time  while  maintaining  a  high  level  of  quality,  planning  and  organization, 
conununication  and  achievement  of  goals.  The  measurement  of  quality  of 
work  of  our  people,  using  these  factors,  appear  quite  consistent  after  TQM. 


H:  Deliberately  aligned  to  support:  continues  improvement,  employee 


partidpation,  recognition  of  TQM  behavior,  customer  orientation,  process 
leadership  and  Improvement. 

I:  Has  measure  and  targets,,  involves  employees,  assigns  employee  objectives 
w/corporate  goals,  focuses  on  quality  as  one  of  the  outputs 

B:  The  cunent  system  is  constantly  scrutinized  to  see  where  process 
improvements  can  be  made 

E:  Incorporation  of  teamwork,  process  management,  customer  focus,  quality, 
involvement ,  and  empowerment 

C;  See  response  to  question  37. 

A;  Management  by  fact,  use  of  quality  toois./measures, 

D:  Have  employees  and  managers  evaluate  and  critique  the  review  criteria 


39.  What  changes  woutd  you  like  to  see  with  regard  to  your  performance  appraisal? 

Organization: 

F;  Remove  form  for  non  exempts  due  to  higher  goal  involvement  and  the  barrier 
it  caused  between  the  two  groups.  More  foais  on  managing  performance  on  a 
day  to  day  basis. 

G.  We  are  working  towards  including  team  based  compensation,  plus  peer 
reviews.  We  are  told  that  both  have  to  be  careful  though  out  before 
implementation 

H;  More  management  training  in  use  of  appraisal  system  (currently  only  2  day 
course). 

I:  More  disciplined  application  across  ail  rriariager/departments. 

B  .  Considering  eliminating  the  actual  assignment  of  a  rating  as  part  of  the 
appraisal  process 

G.  Current  system  of  a  formal,  on-line  system  is  too  bureaucratic 

C  Reduce  complexity  of  forms 

A;  Better  coaching  and  comm  skills  (listening  and  speaking) 


D:  More  emphasis  on  training  managers  and  employees  on  the  review  process  and 
to  discuss  perfonnance.  More  emphasis  on  development  and  future 
performance. 

C  Additional  Comments  and/or  Suggestions: 

Organization: 

G:  In  37  years  of  HR  work  I  have  never  met  a  HR  professional  who  thought  they  had 
a  perfect  appraisal  system.  The  best  I  have  ever  seen  is  one  where  the  emphasis  is 
on  constant  communication  between  management  and  employee.  The  form  is  not 
the  big  factor  more  than  how  it  is  carried  out. 


App«ndix  H:  Presentation  of  Round  Two  Data. 

Delphi,  Round  2.  The  round  two  questionnaire  consisted  of  questions  that  reflected  the 
literature  regarding  the  performance  appraisal  in  a  TQ  environment.  The  questionnaire 
began  by  soliciting  responses  based  on  the  same  Likert  scale  that  was  used  in  the  first 
round.  The  second  section  of  the  questionnaire  dealt  with  the  uses  of  the  performance 
appraisal.  In  this  section,  the  participants  were  asked  to  rank,  in  order  of  importance,  1) 
how  their  organizations  actually  used  the  performance  appraisal:  and  2)  what  they  believed 
to  be  the  best  uses  of  the  appraisal  in  a  TQ  environment.  There  were  eleven  choices  and 
each  participant  had  100  points  to  allocate  for  each  of  the  two  questions  addressed  in  the 
previous  sentence.  Tne  third  bcction  of  this  i  ound  consisted  of  open-ended  questions  that 
sought  descriptive  information  regarding  promotion  decisions,  and  the  extent  of  rater  and 
ratee  training  regarding  the  performance  appraisal.  The  dau  that  follows  is  a  summation  of 
Delphi,  round  two  response. 


The  data  tliEt  follows  is  a  summation  of  Delphi,  round  2  responses. 

The  responses  for  questions  1  through  20  were  based  on  the  following  Likert  scale: 


1 

2 

3  4 

strongly 

neither  agree 

strongly 

disagree 

nor  disagree 

agree 

Organization 

Question 

A 

B 

C 

D 

E 

F 

G 

H 

I 

1 .  Our  organization  rewards/compensates 
based  on  team  performance 

4 

n 

2 

T 

3 

5 

4 

2.  Team  performance  is  a  criterion  which 
is  rated  on  our  performance  appraisal 

5 

4 

4 

1 

4 

X 

1 

4 

5 

3  Continuous  employee  improvement  is 

4 

5 

4 

5 

X 

5 

5 

_2_ 
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the  BoaJ  of  our  appraisal 


4.  Promoting  continuous  process 
improvement  is  a  goal  of  our  appraisal 


5.  OiiT  organization  rewards  individuals 
based  on  individual  appraisal  ratings 


6.  Our  organization  focuses  on  the 
performance  of  the  system  vs. 
erformance  of  the  individual 


is  used  mainly  as  a  source 
e  employee  to  improve 
nance 


8.  Conipensation  is  based  on  an 
accumulation  of  skills 


9.  Compensation  is  based  on  an 
accumulation  of  responsibilities 


10.  Compensation  is  based  on  the 
orcanization/department  success 


1 1.  Our  appraisal  takes  an  employee’s 
basic  competence  for  granted 


12.  Our  appraisal  could  be  characterized 
as  a  coaching  instrument 


13.  The  organization  compensates  based 
on  team  performance 


14.  The  organization  rewards  based  on 
team  performance 


15.  Performance  goals  are  establis.'’  ed  by 
teams 


16.  Participation  in  process  improvement 
teams  is  an  important  element  in  the 
organization’s  appraisal 


17.  The  organization’s  appraisal  is  used 
for  future-oriented  purposes 
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1 9  The  ratee  is  encouraged  to  review  the  {  5  j  5 
completed  appraisal  before  it  becomes  a  | 
matter  of  record  I 


20.  Throughout  the  organization,  raters  |  4 
interpret  the  rating  procedures  uniformly  ! 


4  2|  31  3  3  x 


Construct 


Organization 


used  for  determining 
promotable  employees 


Used  for  making  layoff 
decisions 


(how  it  is  actually  used/expert  opinion  of  how  it  should  be 
_ used  in  a  TQ  organization) 


B  A.  C  lEDHGF 


0/0  10/5  10/1  0/0  20/1  5/10  20/2 

0  0  0 


0/0  10/5  5/5  0/0  5/0 


Used  for  making  dismissal 
decisions 

0/0 

5/5 

0/0 

used  for  future 

10/4 

10/2 

70/7 

development  of  the  ratee 

0 

0 

0 

Used  to  categorize  the 
ratee  based  on  past 
performance 

10/0 

0/0 

0/0 

used  for  making 

50/2 

compensation  decisions 

0 

n 


used  to  foster  employee 
empowerment 


used  to  evaluate 

10/1 

10/1 

0/0 

performance  based  on 
objective  criteria 

0 

0 

used  to  evaluate 

10/1 

10/1 

0/0 

performance  based  on 
subjective  criteria 

0 

0 

1.  Please  Briefly  describe  the  factors  that  are  considered  in  making  promotion 


Organization: 


I.  1.  past  perfonnance  and  accomplishments 

2.  managers  rating  of  readiness  for  promotion 

3.  Employee  assessment  on  key  criteria  established  by  sr.  mgmt. 

4.  EEO  status  code 

5.  Need  development  opportunities  for  key  employees 

A.  below  are  equally  important 
Knowledge  skills,  abilities 
Diversity  profile 
Visibility 

cultural  fit  with  higher  r 

C  Relative  accumulated  skills/knowledge 
Leadership  ability 
Current  Performance 

Right  iob  for  the  person  (development  assignment) 

B.  In  most  cases,  promotions  are  a  result  of  an  internal  job  posting  process.  All 
available  positions  are  first  posted  within  the  company.  Any  employee  is 
eligible  to  post  for  any  position.  An  internal  application  is  completed, 
submitted,  and  reviewed  by  the  employment  manager  and  the  hiring  manager 
to  determine  eligibility.  A  current  performance  appraisal  is  submitted  with  the 
application,  as  well  as  other  resume-type  information.  Both  current  and 
previous  work  e.xperiences  are  taken  into  consideration  when  reviewing  these 
internal  applications.  Several  of  our  jobs  are  set  up  as  “progression"  jobs. 
Certain  skills  are  identified  and  when  those  skills  are  achieved,  the  individual 
progress  or  is  promoted  to  the  next  level.  These  promotions  are  attained 
based  on  time  in  position,  acliievement  of  certain  proficiency  level,  and  not  a 
posting  process. 

E.  30  technical  skills 

20  Soft  skills  (verbal/written  comm.) 

15  leadership 

10  Teamwork  cooperation 
10  contribution  to  goals 
5  performance  ’’  others 
5  ability  to  perform  at  next  level 
5  Part  performance 
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D  This  depends  on  the  type  of  job.  For  non-exempt  jobs  if  meet  minimum 
qualifications,  a  formula  is  used  where  tenure  =  50%  and  performance  =  50%. 
For  exempt  and  managerial  jobs,  job  relevant  experience  and 
education/training  are  both  highly  important.  For  some  non-exempt  jobs, 
performance  and  tenure  serve  to  rank  order  candidates  and  then  experience 
and  education  training  determinde  which  one  ot  the  top  ranked  candidates  is 
selected. 

C.  1.  Qualification  for  job 

2.  Performance 

3.  probabilility  success  in  job 

4.  Leadership 

5.  Education 

6.  Past  ability  to  grow 

H.  Track  record  (performance) 

Expenence  variety 
Personal  goal 

Teaming  capability 

2,  Please  state  how  much  formal  training  raters  receive  regarding  the  appraisal  and  the 
rating  process. 

Organization: 

I.  Zero 

A.  3-4  hour/year 

C  4  hours  -new  supervisors 
2  hour  update  every  tliird  year 

G.  Formal  classes  are  available 

One  day  on  writing  appraisal  and  administering  performance  mgmt  process, 
one  day  on  coaching  and  oroviding  feedback 

D  most  none,  some  get  a  little  as  part  of  a  management  practices  class. 

E.  .\ppraisal  training  is  a  core  topic  in  required  annual  training  for  all  managers 
Managers  receive  16  hours  initial  traimng  in  writing  performance  plans  with 
measureable  objectives,  evaluation  ,  and  face  to  face  feedback  with  employees 
Annually  each  manager  recieves  4  hours  of  refresher  training  With  each 
change  in  the  appraisal  system.  Managers  receive  2  hours  of  training  on  the 
new  system  requirements 


rn 


H.  1/2  day 


G.  1  -2  hours  a  year 

Please  state  how  much  training  the  ratees  receive  regarding  the  appraisal  and  the  rating 
process 

Organization; 

I.  Very  little 

A.  2  hours/year 

C.  2  hours 

B,  4-6  hours  self-  paced  on  writing  defining  objective  for  themselves  and  others 

D.  None 

E.  One  hour  annualy  by  supervisors 

H.  1/2  day 
G.  zero 


4.  additional  comment/suggestions 
Organization: 

C.  Organization  C  at  the  time  of  the  Baldridge  award  was  a  full  integrated  car 
company  with  approximately  10,000  employees.  Since  1992  Organization  C  is 
a  sales  marketing  org.  w  approx.  450  employees.  Eng.  and  Mfg.  are  now 
under  GM. 

D.  I  had  to  interpret  what  some  of  the  uses  in  section  b  meant  and  how  they 
differed  from  each  other.  For  example,  how  are  6  and  1 1  different,  and  what 
does  5  mean. 

E.  Performance  systems  should  accommodate  empowerment  and  teamwork  not 
be  used  as  a  tool  to  implement  them.  Employees  should  assist  and  contribute 
to  defining  performance  objectives. 
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Appendix  I;  Presentation  of  Round  Three  Data. 


Delphi  Round  Three.  This  round  consisted  of  two  sections.  The  first  section  sought  to 
gain  consensus  on  the  questions  from  round  two  regarding  the  participants'  expert 
opinions  on  the  best  uses  of  the  appra-  al  in  a  TQ  environment.  Only  the  questions  that 
did  not  achieve  consensus  from  the  previous  round  are  asked  again  The  second  section 
of  this  round  consisted  of  three  questions  regarding  peer  ratings  and  mutual  goaJ-setting 
These  questions  generated  responses  based  on  the  same  1  through  5  Likert  scale  that  was 
used  in  the  first  two  rounds. 


A.  The  purpose  for  this  section  is  to  attempt  to  gain  consensus  on  these  questions  that 
appeared  in  round  two.  Only  the  responses  that  have  not  gained  consensus  of  the 
participants’  opinions  are  included.  You  have  50  total  points  with  which  to  allocate.  The 
higher  the  number  allocated,  the  more  imponant  its  use  is  in  a  TQ  environment,  Rank 
order  these  uses  for  the  appraisal  based  on  your  expen  opinion  of  the  best  uses  of  the 
appraisal  within  a  TQ  environment.  In  this  section,  we  want  to  know  your  opinion,  not 
the  way  it’s  used  in  your  company 


Construct  Organization 

— —  'll  ■■■!  — I-  .■■■■I.  ■  IP.  ^  ■  . .  ■  , 


Our  apppraisal  is  ... 

;  A 

i 

1 

B 

i  C 

1 

D 

E 

F 

;  ° 

1 

H 

;  I 

1 

Ussil  for  dst£rTTUiiin{j  prornotsbl^ 
employees 

t-- 

'  > 

i 

t 

!  5 

! 

0 

5 

X 

i  10 
! 

0 

i  0 

1 

f 

Used  as  a  tool  for  future 
development  of  the  ratee 

j  10 

1 

1 

25 

i  30 

1 

j 

35 

5 

X 

1  15 

f 

1 

30 

,  15 

i 

1 

Used  for  making  compensation 
decisions 

•  fO 

10 

i  5 

25 

X 

■  15 

10 

.  10 

Used  to  foster  teamwork 

^  5 

5 

:  5 

t 

i 

X 

;  5 

5 

'  12.5 

Used  to  foster  employee 

!  20 

5 

■  5 

0 

10 

X 

'  5 

5 

'  12  5 

empowerment 


B.  Please  respond  to  the  following  questions  based  on  the  scale  below. 


strongly  n 

disagree  n 

Construct 


Our  organization  uses  peer  ratings  as 
opposed  to  supervisor  ratings 


neither  agree  strongly 

nor  disagree  agree 

Organization 
ABCDEFGHI 


s  :  l;  11  li  11  2!  xl  ll  3!  1 

M  I  i  !  I  I  1  I 

;  I  !  I  !  I  :  !  ; 


1141  xl  2|  51  4i 

'  i  I  '  1  i 


Performance  goals  are  mutu'  '''  •  agroed 
upon  between  rater  and  ra-  ^ 
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